As governments, state archives and libraries struggle to preserve and provide access to digitized government records and cultural heritage-related resources, the need for long-term digital storage is finally beginning to receive the attention it deserves. In today's Internet-based environment, a variety of digital storage projects are moving from the theoretical realm into reality. As digital archives become more ubiquitous, the digital preservation profession is developing methods to certify and standardize their quality. At the same time, the archival community continues to struggle with finding acceptable formats for long-term preservation and obtaining ongoing funding for the exploding number of existing and proposed projects.
These themes were discussed at IPRES '06 (http://ipres.library.cornell.edu/program.html), which brought together 220 digital preservation specialists, records managers, institutional repository developers, archivists, and librarians from around the world. The event, which took place at Cornell University on October 9-11, 2006, featured high-level speakers from prominent digital archives, governments, publishers, and libraries.
IPRES '06 was sponsored by the Cornell University Library, a long-time leader in the field of digital preservation and digital library development (http://library.cornell.edu/about/digital.html). Cornell also is co-developer with the University of Virginia of the Fedora open-source digital object repository system, and home to the renowned Cornell Lab of Ornithology's Macaulay Library of digital wildlife recordings (see Appendix). The conference received support from JSTOR, Microsoft, OCLC and Sun Microsystems.
Merging the Role of Library and Archives: The Canadian Model
There still is a divide between the library and preservation fields, with libraries leaning towards collection development and providing access, and archivists focusing on long-term preservation. In many respects, however, Canada is setting a new model for libraries and archives to work together, rather than in their traditionally separated roles. Two years ago, Canada's national library and archives became one organization, a fusion that Ian Wilson, Librarian and Archivist of Canada, believes meets the needs of Canada's government and public. "The public does not care about silos and professional differences between librarians and archivists," he said, adding that information professionals must tap into popular interest and not just "talk among each other."
Wilson also discussed Library and Archives Canada's (http://www.collectionscanada.ca/index-e.html) twin goals to preserve Canada's heritage and public record. He cited the 1990s as being the "worst documented decade in [Canadian] history," due to the transition between a paper- and a digital-based records system. Now, he said, digital is no longer viewed as an ancillary means of preservation: Digital technologies form the basis of how Library and Archives Canada builds, acquires, preserves and provides access to collections. On the downside, he highlighted the huge future expenditures that will be required to maintain and keep online large-scale digital projects.
Archivists continue to consider which digital formats to adopt for preservation. On the one hand, DVDs or web repositories with compressed files are not considered robust enough for preservation purposes. At the same time, they are useful formats and platforms to provide broad access to materials and collections to users. However, with more and more items being born digital, this issue will become less and less important, in the sense that the digital format of objects will be embedded in their creation.
As the world continues its rapid transition from analog to digital, archivists in memory institutions still grapple with the dilemma of reformatting analog source material. In the specialized area of audio-visual collections, there is a growing acceptance of digital files as preservation copies, said Carl Fleischhauer, Project Coordinator, Office of Strategic Initiatives, Library of Congress. This practice is reasonably well established for sound recordings but still emergent for moving image content. For example, many preservation projects use WAVE or Broadcast WAVE as a target format for audio, but there is not yet a consensus for video. In the face of this uncertainty, many video reformatting projects adopt a "hybrid" approach. The preservation master is recorded onto conventional videotape, e.g., DigiBeta, while lossy compressed computer files, e.g., MPEG-2 or -4, are made for day-to-day viewing. Meanwhile, Fleischhauer noted, record companies, television broadcasters, and theatrical film producers have embraced digital technologies for production, although archivists in the commercial sector are troubled about how best to manage all of this new digital content.
Fleischhauer also highlighted the file-storage dilemma faced by archives that cannot afford large-scale server systems. These organizations often use CDs and DVDs for content storage, in spite of their awareness of short media life expectancies. This practice, Fleischhauer said, highlights the need for consortial storage arrangements or the provision of trustworthy and affordable third-party storage services, one of the topics being explored by the National Digital Information Infrastructure and Preservation Program at the Library of Congress.
Quantifying the Quality of an Archive
There is a growing movement underway to develop metrics that measure the quality or "trustworthiness" of an archive. Proponents of certification believe the process will create more standardized, reliable and credible archives that better meet the long-term needs of libraries, governments, and user groups.
Susan Dobratz, head of the joint Electronic Publishing Group of the Computer and Media Services at the University Library of Humboldt-University Berlin, discussed Germany's Nestor (Network of Expertise in Long-Term Storage of Digital Resources) project (http://www.langzeitarchivierung.de/index.php?newlang=eng) that aims to set up "criteria for trusted digital repositories [and] recommendations for certification procedures of digital repositories." As more archives come online, she said that "users want guidelines and coaching on how to set up a trustworthy archive."
Robin Dale, Program Officer, OCLC Office of Programs and Research, discussed the work of the RLG-NARA Digital Repository Certification Task Force, as well as the Center for Research Libraries Auditing and Certification of Digital Archives project (http://www.crl.edu/content.asp?l1=13&l2=58&l3=142) funded by the Andrew W. Mellon Foundation. Dale said the audit process in many ways is more important than the actual certification, since it allows archives developers to analyze and respond to their archives' strengths and weaknesses in a systematic fashion. Leveraging the audit checklist developed by RLG and NARA, the CRL project conducted several pilot audits, informing further checklist development. Many preservationists in the audience were familiar with the RLG-NARA audit metrics, having used them as a "checklist" during the creation of their archives (http://www.rlg.org/en/pdfs/rlgnara-repositorieschecklist.pdf).
Growing Number of Projects
Many impressive projects, including e-journal and international repositories, were profiled at IPRES '06. Their scope, technological development, and business models demonstrate a growing sophistication.
Libraries continue to rely on e-journals provided by subscription from publishers. As they discard paper holdings, long-term access to back-issues in electronic format has become crucial. As publishers change their holdings, subscription terms, and business models, libraries are often left in the dark as to whether complete archives of journals will be available in the future. To meet this need, the number of e-journal preservation archives is growing. Such repositories are generally dark (or not accessible), provide back-up to publishers in the event they cannot or do not publish e-journals, and are supported by grants, governments and/or institutional membership. Several e-journal archives were profiled at IPRES, including:
The large number of international projects presented at IPRES makes it clear that many countries are developing impressive digital archives, including Switzerland, the Netherlands, the United Kingdom, and China. Often repositories are developed by national libraries or large universities, although each country is using a different model for financial support and technical format, and much can be learned by examining these projects. Details on these and many other projects and digital preservation developments can be found in the IPRES speaker presentations (see http://ipres.library.cornell.edu/program.html).
Setting the Stage for IPRES '07
In many respects, the digital preservation profession continues to face the age-old archival dilemma. Although many projects are becoming more high-profile, developers continue to struggle to obtain ongoing funding and find it difficult to create a commercially successful business model. Like digital libraries in general, most digital archives are supported by grants, direct contributions from parent organizations, or government subsidies. The financial picture is further complicated when the projects receive one-time funding to establish themselves, and then must find additional support on an ad-hoc basis.
At IPRES '06, however, there was a buzz of excitement, as the digital side of the preservation field is becoming more necessary in today's e-environment. Governments, national libraries, universities, and commercial enterprises not only recognize, but support, endeavors to preserve documents and resources for future generations. Concurrently, many archives have achieved or are building greater levels of sophistication, standardization and technical complexity into their projects.
"Having 200 plus folks show up in centrally isolated upstate New York for a conference solely devoted to digital preservation is a tribute to how far we've come in the past several years," said Anne R. Kenney, Senior Associate University Librarian for Public Services and Assessment at Cornell. "I think we can look to more conferences on the topic that present cutting edge initiatives and more opportunities for birds of a feather to get together to discuss particular issues in digital preservation." No doubt IPRES '07 will provide the setting to discuss digital preservationists' challenges and successes in the e-era.
Appendix: Cornell Lab of Ornithology's Macaulay Library
One of the world's most unusual digital archives is housed at Cornell: The Cornell Lab of Ornithology's Macaulay Library (http://www.birds.cornell.edu/MacaulayLibrary/About/index.html). The Macaulay Library is an impressive facility, with technical resources and capabilities rarely seen in the digital library and archival environment. The Macaulay Library contains 160,000 recordings and 3,000 videos of birds and animals from around the world, including the sounds of many species that are now extinct.
Forty people work at the Macaulay Library, including eight archivists, who manage, annotate, and preserve recordings in myriad formats dating back to the 1930s. The library depends on the work of scientists, many of them graduate students, who record animals in the field. To standardize metadata, the library developed free software that requires necessary fields to be filled out, such as time, type of equipment, etc., to better ensure the scientific quality of the data pertaining to each recording.
The Macaulay Library, however, is much more than a digital repository. It is home to a vast array of production and post-production studios and equipment, climate controlled archives, and technical staff who are constantly improving the quality and distribution of its holdings. For example, explains Media Integration Developer Guillaume Iacino, the web repository will soon be reconverted from a Real-based platform to MPEG-4 to allow more options for users to playback the recordings. The number of HD videos is growing, and soon the library will unveil an image collection.
With sustainability in mind, the archive has made a conscientious effort to diversify its income stream selling sounds in a business-like fashion while remaining dedicated to providing free or low-cost access for education and research purposes. The library has many innovative ideas for revenue generation, from selling stuffed animals with embedded sounds of the actual animals to promoting the sales of licenses to commercial companies. The library, explains Mary Guthrie, Production Manager, operates on a cost-recovery basis. Research and education uses of the recordings are subsidized, and the library helps recoup its costs through institutional and commercial licensing, along with a variety of commercial ventures and partnerships, from toys to endowments.
The library is committed to expanding the reach of the archive, says Greg Budney, Interim Director of Macaulay and Curator of the Audio Collection. "We want to become readily available to everyone," he says, highlighting the need to reach developing regions in particular. "For example, we want conservation forest managers in Ghana to access the sounds and recordings of wildlife from Ghana." As the organization continues to grow, he emphasizes, the archives will continue to serve in their primary role as a research collection for education and conservation.
Copyright © 2006 Cindy Boeke