Rehousing digital heritage. Preservation on a very large scale

BOER, Tanja de and SCHIE, Maarten van and SIERMAN, Barbara and WESENBEECK, Astrid van (2014) Rehousing digital heritage. Preservation on a very large scale. Paper presented at: IFLA WLIC 2014 - Lyon - Libraries, Citizens, Societies: Confluence for Knowledge in Session 188 - Preservation and Conservation Section. In: IFLA WLIC 2014, 16-22 August 2014, Lyon, France.

Bookmark or cite this item:
Language: English (Original)
Available under licence Creative Commons Attribution.


Rehousing digital heritage. Preservation on a very large scale

More than ten years ago the e-Depot of the National Library of the Netherlands (KB) became operational; one of the first long-term archives for international scientific publications, worldwide. Millions of publications, mostly e-journal articles in PDF format, were ingested, stored and made accessible. For digital preservation and for scaling purposes the KB started in 2009 renewing its digital storage environment and building and equipping a new digital preservation system. A first release of the newly built system was dedicated to receive the millions of stored articles and to process the migration which would phase out the initial e-Depot. We could then start migrating collections to a new, better and future proof storage. To move our digital collection, we needed to migrate these objects to the new hardware- and software environment.For KB, migrating millions of publications meant a major preservation action on an unprecedented scale. And one with possibly great risks for our digital heritage when not properly executed. It was not only a media migration. Almost 5 million publications had to be repackaged, which meant all files and metadata were checked and reformatted into new archival packages. We developed an automated workflow to process the material as well as processes for quality control and error handling. The entire operation took 10 months and cost around €110.000. Almost 20 KB-staff members ensured a controlled and safe preservation process. Two members of staff were dedicated to monitoring the daily progress of migration and report on any errors. We set a fault tolerance for the migration of the files, and the project had to meet a strict deadline. After careful preparation and testing the actual migration started in January 2012. We chose a two-step approach. The first step was migrating collections from the old e-Depot environment to temporary storage on a secure environment ‘silent-cube’. In the second step, collections were transferred from temporary storage to the new digital preservation system. This two-step approach limited our dependency on the relatively poor processing capacity of the old e-Depot system. Migration was concluded in October 2012, with exception of the event metadata. At that point almost 5 million objects were successfully migrated through the two steps to their final new location and were instantly available to our patrons. KB has greatly learned and profited from this project. We have gained experience in executing a complex preservation action on a large scale. And that will help us planning and executing future preservation actions, and also on more complex digital objects. We hope to share our lessons learned with fellow libraries as well as learn from their experience.

FOR IFLA HQ (login required)

Edit item Edit item