Currently, the general practice of our three publishing partners is to regard their electronic titles as parallel (and possibly supplemented) manifestations of their print editions. Although all three have indicated a willingness to explore the modality of issue-less publishing at some point in the future, none is actively experimenting with the concept. Since the notion of issue-less journals represents a major shift in serials publishing practice and is unsupported by current library systems and scholarly practice of citation, we have decided to retain the concept of the issue as central to the design and implementation of the Archive. However, we retain some flexibility in this matter by allowing a loose definition of issue as a publisher-specified aggregation of content items without necessary regard for fixed publication patterns. As will be apparent in Section 4, this issue-centric focus is central to the design and implementation of the Archive. From the perspective of ingestion, this issue-centric focus allows Harvard to control receipt of content and to determine by examining the sequence if something has not been received. Ingest will be based on our publishing partners depositing new e-journal content in the Archive on a predefined schedule. The suggested schedule, each issue to be deposited prior to release of the next issue, will allow for an even and manageable ingest flow.

Many think of e-journal archiving only in terms of preserving articles. However, e-journals actually contain a complex range of components. The target for preservation is defined in this project as the electronic version of the journal. At this early experimental stage, it was deemed best to preserve as much as possible in the categories of components and functionality. To determine what components and functionality of e-journals exist, twenty-one journals were examined (see Appendix B). This sample included eleven titles from the publishing partners and ten titles from other sources. They covered a wide range of disciplines and represented titles available in print and electronic formats as well as titles only available electronically.

For all journals with printed versions, information about both the printed and electronic versions is available in the electronic version. All journal sites have basic descriptive information such as scope and purpose, subject coverage, and copyright statement. Most sites also have ISSN, frequency statement, indexing and abstracting service coverage, current editorial board, submission information, subscription and reprint information, and contact information. This category of information is equivalent to the front matter of the printed journal and provides an essential intellectual infrastructure for the journal. In discussion with the publishing partners, it has become evident that much of this front matter is not preserved in the electronic environment: only the most current version is available. While the editorial board information is associated with a particular issue in print versions of journals, this is not the case in the electronic version. Linking the appropriate version of front matter to specific issues is now an important item for the publishing partners to explore.

Within the issue or discrete publication bundle, all journals have table of contents information. Items listed in the table of contents include articles, case reports, comments, communications, correspondence, responses, dialogue, columns, editorials, letters to the editor, book reviews, conference notes, news, announcements, interviews, errata, volume indexes, subject indexes, membership lists, and reviewers.

Advertisements are found in two of the journals reviewed. Advertisements present a particular challenge for archiving. First, it is not uncommon for Web advertisements to be served from an organization other than the content publisher so that archiving agreements would not necessarily cover their deposit. Second, advertisements are frequently "dynamic," changing from day to day. The same page viewed on different days can have different advertisements. The advertisement seen in one country may be different from that seen in the same context in another; drug advertisements, for instance, are regulated at the national level and therefore vary with the country of receipt. What is the appropriate advertisement to archive with a given issue? When would dynamic advertisements be archived?

Web advertisements will be an important source for documenting contemporary business, society, design, and technology. However, they represent a minor type of content for scholarly e-journals. Harvard has decided not to archive advertisements as part of this e-journal archiving project. We hope, however, that someone somewhere is archiving Web advertisements more generally as part of the documentation of our time.

For journals examined in this survey, most articles include an abstract in HTML. Generally, the articles are delivered in both HTML and PDF; however, other formats noted include Postscript, TeX, and DVI, delivered as individual files or as aggregations bundled together in ZIP packages. The HTML versions of article content offer thumbnail images of tables, generally in GIF format and occasionally in JPEG format. Tables are also included in the HTML versions. Figures, equations, symbols, and other graphics are delivered in GIF and JPEG formats.

One of the great powers of digital journal articles is that they are not limited to linear text and static pictures. Increasingly, articles include "supplementary materials," digital files of many types. These files can include digital materials used in the research described (statistical or instrumentation datasets, for example) or materials that expand on or illustrate topics discussed in the article (simulations, or tables too large for inclusion in the base article, for instance). These supplementary files represent a significant resource but also a significant challenge to the Archive.

In general, there is little control over the technical formats for supplementary files, no guidance to authors about good practices in the creation of such files, and little editorial analysis of the file content. The technical heterogeneity of these materials could introduce a wide and ever-growing range of formats into the Archive, significantly increasing the complexity of the preservation task. The lack of guidelines and quality control means that unlike the case of articles themselves, the Archive is faced with objects of unknown virtue and potentially troublesome content.

One of our publishing partners suggested that it would be very useful to publishers if archives could provide guidance on preferred technical formats and practices that are well suited for preservation and archiving. In the current environment, there is nothing to suggest to authors how to create digital objects with a greater chance of long-term viability. Many authors and their editors are very concerned about the longevity of their publications and if given guidance, may well be willing to change practices in ways that will reduce the complexity of preserving supplementary article content.

Most articles offer internal linking among components within the HTML version of the full text. External links are also common and link out to authors' email addresses, author-provided URLs, citations in external indexing resources, other articles by the same author, and related articles. Linking is a fast-changing area in e-journals and represents one of the great value-added features of electronic over paper publishing. However, links pose significant challenges in archiving. The largest application of links today is for references, allowing a user to navigate automatically from one article to a related cited article. Most publishers, however, do not simply insert static links in references. Most links today are in the form of Digital Object Identifiers (DOIs), a type of persistent link or "name" that remains valid even if the cited work moves between systems or publishers. Frequently publishers keep a database of references and only determine the DOI for a citation -- a costly process -- once. This allows the DOI to be reused each time the same work is cited. Further, the number of DOIs for retrospective articles is growing very rapidly so that even if no link were available for a reference when an article is originally published, one could be added later when the DOI becomes available.

While the majority of links found in e-journals today are for references, there is every reason to expect linking to become richer with time. There is already significant growth in links to "knowledge bases" such as GenBank.[4] As these new types of links occur they will be added by automated means to existing articles.

Because links are dynamic and are expected to grow with time for already archived articles, it is unclear whether an archive should attempt to capture those links available at the point an article is ingested. Would it be better for archives to implement the types of dynamic linking systems that publishers use, allowing for ever more rich links for archived content? Or, should archives arrange for publishers to periodically resend the links for articles submitted earlier? In either case, there will be complexity involved in supporting links for archives, but links are a vital type of content and users will likely be dissatisfied if they are not included in archived content. This is an area requiring more exploration.

All journal Web sites include browsing functionality and most include search capability. All journals except one include help features.

As we considered which components of electronic journals should be archived and how much of the look and feel of an e-journal should be preserved, there were concerns that costs would prohibit comprehensive archiving. It is clear that all articles, reports, columns, editorials, communications, abstracts, errata, and correspondence must be archived. It is less clear which other components should be preserved and how nontraditional components -- such as links, threaded discussions, data sets, and data simulations -- should be handled. To address these questions, we designed a survey that we completed by interviewing faculty in the sciences (see Appendix C). The survey focused on journal functionality including browsing, searching, image size and content (including cover images), tables of contents, subject and author indexes, advertisements, editorial board membership, editorial policy, announcements, membership lists (for societies), reviewer lists, copyright, guidelines for authors, career/job information, and business information (advertising guidelines, subscription information, and contact information). Faculty were primarily concerned with the reliable archiving of scientific content, specifically articles, reports, editorials, and other original content, plus functionality, including browsing, searching, and printing. Hierarchical links among volumes, issues, tables of contents, and articles were identified as important. Threaded discussions were of interest, but not considered critical by some faculty since they are not peer-reviewed. Access to original data sets provided by the authors was also considered useful, although providing reliable and accurate links to materials not maintained by publishers is problematic.

Based on analysis of the e-journal sample and the user survey, Harvard has defined a preliminary list of materials deemed to be in the scope of archival collection and those currently out of scope of the archival collection, and will work with selected publishers to identify which components are available. Deposits will include not only journal articles, but also associated materials (e.g., references, external links, abstracts), author-created supplementary digital files (e.g., datasets, sound files, simulations), other editorial journal content (e.g., editorials, reviews, communications, letters, threaded discussions), and selected masthead information (e.g., editor, editorial board, copyright statement). Materials currently defined as in scope should be deposited, while those defined as out of scope are not expected but may be deposited if available, with the exception of advertising which will not be accepted.

The following components should be deposited in the Archive:

The following types of components are not expected or required for deposit in the Archive:

Harvard originally proposed that the Archive should initially be semi-dark, permitting access only to Harvard University Library's authorized users through an online process and to any user legitimately authorized by the publisher through a batch export process. Such access would allow for maintenance, auditing, and minimal exercising of the data. The publishing partners had some concerns about this position including:

During the resulting discussions, Harvard agreed that throughout the dark period the Archive would be accessible only by its operators and by a designated outside auditing authority. Under certain circumstances, archived material might be made available to authorized users who can present proof of their legitimate right to specific materials.

The Harvard University Library Access Management Service (AMS) allows fine granularity of access control, down to the level of permitting access to a particular object by a particular user through the use of a particular application. This level of control is appropriate and practical to support auditing, but expensive to extend to large-scale external use. An archive intending to restrict access to only those who have had a past subscription to the archived content would bear considerable expense to gather the required licensee information and to build and operate the appropriate access management system. Rather than pursue this option, Harvard believed that the increasingly wide-spread adoption of the "moving wall" concept (in which content becomes publicly available after a specified time period) in the scholarly journal environment suggested a more practical approach. After deposit, archived materials would enter an initial "dark" period chronologically bounded by the trigger events defined in the submission agreement. When any one of the trigger events has occurred, material would be accessible without restriction.

At various times over the planning period, the following possible trigger events (conditions which would cause archived content to become publicly available) were discussed:

Trigger events are one of the key provisions of an archiving plan. They define when the preserved content, which someone has made a considerable investment to archive, is useable. Because they touch on areas that affect the commercial value of the content and thus on the publisher's income, publishers are legitimately quite concerned that they be carefully constructed. All parties to archiving -- authors, publishers, archives, subscribing libraries -- have an interest in the details of trigger definition. This is an area requiring further discussion among the concerned parties.

The Ingest function is responsible for accepting and acting upon submissions of material for deposit into the Archive. The technical infrastructure necessary to support the following subtasks within this function are for the most part not currently extant, and their implementation would occupy the majority of the first year of an implementation project.

The Data Management function is responsible for maintaining descriptive information about archival holdings and administrative data necessary to the internal management of the Archive. Issue and item-level descriptive metadata is received from the Ingest function. Issue-level metadata is stored in the existing HOLLIS ILS, which includes serial check-in and claiming mechanisms useful to detect and request submission of missing issues. Item-level metadata is stored in a new catalog, implemented either in the HOLLIS ILS or as a stand-alone XML database application.

Our three publishing partners currently offer 1,137 electronic journals, annually comprising over 210,000 articles with a total size of approximately 400 gigabytes per year (assuming SGML/XML full text and TIFF images for articles, each with an accompanying PDF file). They project relatively modest growth in their electronic offerings over the next five years, with an anticipated increase in titles of three percent per year.

The current storage architecture underlying our operational Digital Repository Service (DRS) uses NFS-mounted, RAID-based devices as its primary online storage mechanism, with automatic replication over a dedicated T1 network to an off-site tape library which can be mounted as a file system for remote recovery access. The operational policy for the tape library enforces automatic periodic tape refreshment on a five year schedule. The total growth capacity of the current implementation of this system is 50 terabytes. The recovery cost for storage under this system is $20/gigabyte/year.

The current practice of our publishing partners is to present journal content in the form of static text and visual images. As advanced dynamic media types, such as streaming audio and video, become more prevalent in electronic publishing, the per-issue size requirements of the Archive will increase commensurately.

The Archival Storage function of the Archive is provided by the extant DRS. The DRS batch loading process requires that each physical data stream be available as a separate file, along with an additional XML-encoded control file specifying loading and storage options. This proscribes the form of the AIP within the DRS as the complete set of SIP components with each issue and item-level METS metadata and content file deposited as individual digital objects. The Ingest function is responsible for transforming the SIP into the AIP prior to DRS deposit. After successful deposit of an AIP into the DRS, the Archive will generate and transmit an e-mail message of confirmation to the submitting content provider. The issuance of this confirmation constitutes the formal notice of the archive's assumption of archival responsibility for the deposited material.

Issue and item-level METS metadata files contain internal pointers to their component content files. In addition, the DRS has its own explicit mechanism to maintain typed relationships between individual digital objects stored within it. Thus, issue and item-level METS files will be stored within the DRS with an inter-level parent/child structural relationship. Similarly, METS metadata files and their component content files will be stored with an intra-level parent/child relationship.

Given the substantial number and size of e-journal components that will be deposited over time, we will allocate resources to evaluate the DRS's scaling properties and, if necessary, to design and implement appropriate enhancements.

Because the purpose of this Archive is to preserve the significant intellectual content of journals -- not the original form in which the content was authored or delivered -- the Archive will most likely rely upon transformation to prevent obsolescence.[15] As defined by OAIS, transformation refers to "a [type of] digital migration where there is some change in the Content Information or PDI bits while attempting to preserve the full information content."[16]

Files that are proprietary and therefore not amenable to transformation will also be accepted into the Archive, provided they are not essential to the meaning of the journal article. The Archive will accept, store, locate, and deliver these files; the Designated Community would assume responsibility for transformation.

The Archive's preservation policies and management functions are format-specific, envisioned to meet the following objectives:

The Access function encompasses both e-journal resource discovery and delivery. Discovery takes place at the title and issue level through the existing Web-accessible HOLLIS OPAC. Actionable issue-level links provide users with a table-of-contents-like view of all individual issue items. Actionable item-level links from the table-of-contents-like issue display and from search result records in the item-level catalog provide users with access to individual issue items. XML-encoded full-text content items are dynamically transformed into HTML using XSLT. Other item content media types (e.g., streaming audio or video) are delivered via the appropriate DRS delivery applications.

Rather than browsing or searching these Archive catalogs for relevant content, a user may possess a priori appropriate citation information for the desired content, such as author and title, chronology and enumeration, or a DOI uniquely identifying an item. A valid access request to the Archive can be made by using this citation information in a properly formed OpenURL which encodes the citation data into an actionable URL.[19] The Archive will implement an OpenURL service to accept such requests and respond with the appropriately displayed item.

Access authorization is a binary function. During the initial dark period following submission, e-journal content is available only to Archive staff for internal administrative and maintenance purposes, and to auditors as described in the Administration function. Once content is lit up in consequence of an appropriate trigger event, that content is available to the general public. Authorization is enforced only at the point at which the delivery of actual issue or item-level content is requested; cataloging metadata for all content is always available to the public.

Delivery of raw e-journal issue data (i.e., issue content and metadata as preserved in the Archive) can be requested and returned as an OAIS Dissemination Information Package (DIP). At least one form of the DIP should be equivalent to a Submission Information Package (SIP) so that a DIP delivered from a compliant archive can be ingested directly by another archive. The Archive will consider support for additional standard DIP formats as they emerge in the future. For the wholesale batch dissemination of archived content, that content will be transformed from its internal Archival Information Package (AIP) form to a DIP. The handling of and response to requests for a DIP will be an off-line, asynchronous operation. It is highly desirable that the archiving community cooperates in the development of standard definitions of the DIP to facilitate the transformation of archival materials between participating institutions.

The Administration function is responsible for the routine operation of the Archive. A good deal of this work is manual, not automated, including the negotiation of submission agreements with content providers, supervision of the Archive staff, and the maintenance and enhancement of the Archive's technical environment and infrastructure. Nonetheless, these manual procedures will be supported by systems providing administrative database services, online registration of content provider profile information, and hardware and software monitoring tools.

The major automated task of this function is the performance of required format migrations as instigated by the Preservation Planning function. The Archive's adherence to the internal use of a limited set of normative Level One data formats constrains what would otherwise be a potentially intractable undertaking into a feasible task. We will investigate the use of commercial tools to transform non-normative formats into normative data formats eligible to receive Level One preservation services. Additional tools may be identified to perform other data management functions, such as validation and metadata extraction, that would assist preservation monitoring when transformation is not feasible. In addition to the transformation and validation of the e-journal content objects, such migrations may also necessitate enhancements to delivery systems. As the set of normative formats grows or is culled over time, ingest procedures will also require concomitant modifications.

Although extensive content quality assurance occurs upon Ingest, subsequent periodic auditing of archived material will also be carried out to validate the lossless nature of migration transformations as well as the general stability of the Archive's storage environment. This auditing will occur under the purview of the Administration function by domain experts drawn from publishers, scholarly societies, and librarians. Statistical sampling of the Archive's holdings categorized by publisher, subject area, and title will be employed in the selection of material to ensure appropriate representative coverage in the auditing process.

The primary responsibilities of the Archive staff with regard to technical issues are the initial development, ongoing maintenance, and future enhancement of Archive systems. Additionally, the expertise of the staff will be helpful in monitoring technical innovation and obsolescence with regard to normative data formats and potential preservation migration transformations. This activity would be performed in cooperation with the Archive preservation manager and collection curators. All technical work will be performed using accepted industry standards and processes for technical management, design, implementation and testing methodologies, configuration management, and documentation.

The Archive architecture relies heavily on efficiencies achieved by compliance to common standards, e.g., SIP structure and the archival schema. Widespread compliance is best achieved by insuring that these standards are developed in an open collaborative process. Archive staff will be responsible for the coordination of these processes and the resultant timely publication of appropriate specifications.

As Archive submission is opened to publishers beyond our initial partners, we anticipate working with institutions with widely varying technical resources and competencies. Potential difficulties in this regard can be mitigated by the distribution of appropriate utilities and tool sets to facilitate publisher activities. All Archive development will be evaluated in terms of the applicability of new system components for such distribution. If found to be relevant, development will proceed with due consideration towards platform independence of the system implementation. Archive staff will also be available for limited technical consultation regarding publisher development and operational procedures.

Harvard's initial approach to archiving has been "publisher based," that is, oriented towards archiving all of the e-journal output of a publisher. This approach was chosen for two reasons. First, it simplifies the task of creating a large base of archival activity to test systems and operations, and to provide enough scale on which to base long-term cost projections. Second, we believed that the marginal cost of archiving another title from a publisher with whom the issues of interoperation have already been worked out would be less than taking a new title from a new publisher. The cost of archiving a title could thus be lowered.

Our plan is to work with our three publisher partners, pending final agreement, to build and test interoperation between the Archive and the publishers' systems, then begin to increase the number of titles archived from each. One of the great uncertainties of archiving is the amount of labor required for ingesting new content. The Archive will very likely be limited in staffing in the initial phase of its operation. We plan to increase Archive coverage until we are ingesting all of the content from our original partners, or until the available staff cannot deal with additional input. At the point where all available titles from the original publishing partners have been deposited successfully and it is determined that we have not yet reached our capacity to ingest content, it will be appropriate to evaluate the Archive's procedures and functions and determine growth options and extended partnerships.

In traditional preservation, curators are responsible for ensuring that collections remain usable. While they may store collections in centralized, environmentally-controlled storage facilities, or partner with conservators and preservation technologists to repair or copy materials, curators are ultimately fully responsible to account for the extent, condition, and usability of their collections.

The inherent fragility and complexity of digital collections require a shift in preservation responsibilities. To ensure that items do not become obsolete, curators and other owners need to delegate preservation responsibilities to technical staff with the expertise and the tools. In this new model, preservation technologists assume perpetual rather than temporary custody of the physical objects in their care. They must monitor the items as well as the environment. Because obsolescence is inevitable for all digital formats, they must be able to develop and present migration strategies to the curators and owners, then implement the strategy the owner prefers. In traditional preservation, curators have ongoing custodial responsibility (whether passive or active) and preservation technologists intervene infrequently. In the preservation model for digital archiving, repository managers assume the ongoing custodial responsibility and curators are consulted as necessary to make decisions about migration and other issues.

Discussion among Harvard and its publishing partners has centered on who "owns" the Archive and who governs it. While the Archive is intended to be maintained and administered by Harvard and built on Harvard's existing digital library infrastructure, the publishing partners have suggested that a broader group with a vested interest should be involved. Who are the stakeholders and what is their role in helping the Archive do its job? This community could comprise authors, scholarly societies, publishers, and institutional subscribers as representatives of researchers. These delegated stakeholder groups should have a role in reviewing the policies and practices of the Archive as a mechanism for vetting the Archive and establishing a level of trust; however, some publishing partners have suggested that actual governance of the Archive is tied to the brightness of the Archive and the additional services that might be offered by the Archive. The brighter the Archive, the more governance a publisher should have; the more services offered, the more governance a publisher should have. Harvard, however, maintains that final governing authority is intrinsically tied to the ability to use its existing infrastructure as a starting point for the Archive, while a variety of policies and procedures related to the development, administration, ongoing maintenance, and financing of the Archive should be developed in consultation with and open for review and comment by representatives of this stakeholder community.

In addition to the stakeholder community with its representative input, we have elsewhere in this paper discussed the necessity for an external auditing service. Such a service might be part of a broader confederation of archival organizations and stakeholders. Such a confederacy might be charged with establishing registries for content and format types; certifying policies, practices and procedures; and supporting the ongoing development of digital archiving.

During the development of the Archive several pieces of sharable infrastructure will be produced:

123Cover image for issue123Table of contents123Volume/issue number linked to content123References to outside information (e.g. portals, author-developed data that is stored outside the journal site, bibliographies developed by the journal, etc.)123Threaded discussion123Index to volume123Subject123Author123Advertisements123Announcements (e.g., events)123Editorial board123Editorial policy123Membership list (e.g., for societies)123Reviewer list123Copyright123Licensing information123Guidelines for authors (e.g., manuscript preparation and submission)123Business information123Advertising guidelines123Subscription information123Customer Service123Contact information123Career/job information

Browsing123Chronologically123By subject123Links within the journal, volume, issue (e.g., article to article links and links to supplementary information within the journal)

Searching123Author123Title123Keyword123Limit by date123Help123View thumbnail of image123View full-size image

The tension that underlies digital preservation issues is the fundamental human tension between mutability and immortality. The ancient Platonic philosophers thought that divine nature was unchanging and immortal and human nature was changeable and mortal. They were right about the human nature at least.

Human products in the normal course of affairs share the fates of their makers. If they are not eradicated, they are changed in ways that run beyond the imagination of their makers. Stewart Brand's How Buildings Learn[1] has lessons for all those who imagine they are involved in the preservation of cultural artifacts, not just people concerned with buildings.

But a very limited class of things has managed a special kind of fate. The invention of writing and the development of cultural practices associated with it has created a unique kind of survival. It is rarely the case that the original artifact of writing itself survives any substantial length of time, and when it does, that artifact itself is rarely the object of much active reading. Most of us have read the "Declaration of Independence" but few do so while standing in front of the signed original in the National Archives.

Written texts have emerged as man-made artifacts with a peculiar kind of near-immortality. Copied and recopied, transforming physically from one generation to the next, they remain still somehow the same, functionally identical to what has gone before. A modern edition of Plato is utterly unlike in every physical dimension the thing that Plato wrote and yet functions for most readers as a sufficient surrogate for the original artifact. For more modern authors, where the distance between original artifact and present surrogate is shorter, the functional utility of the latter is even greater.

That extraordinary cultural fact creates extraordinary expectations. The idea that when we move to a next generation of technologies we will be able to carry forward the expectations and practices of the last generation blithely and effortlessly is probably widely shared -- and deeply misleading. The shift from organic forms of information storage (from papyrus to animal skin to paper) back to inorganic ones (returning, in silicon, to the same material on which the ancients carved words meant to last forever and in fact, lasting mainly only a few decades or centuries) is part of a larger shift of cultural practices that puts the long-term survival of the text newly at risk.

Some of the principal factors that put our expectations in a digital age at risk include:

Against all of these entropic tendencies lies the powerful force of expectation. Our deepest cultural practices and expectations have to such an extent naturalized ideas of preservation, permanence, and broad accessibility that even the most resistant software manufacturers and anxious owners of intellectual property will normally respond positively, at least in principle, to concerns of preservation. That is a great advantage and the foundation on which this project is built.

The reader is the elusive center of our attention, but not many readers attend conferences on digital preservation. We find ourselves instead working among authors, publishers, and libraries, with occasional intervention from benevolent and interested foundations and other institutional agencies. The motives and goals of these different players are roughly aligned, but subtly different.

All of the above tantalizing complications of long-term electronic archiving drew us into this planning project and continued to be themes throughout the year-long planning process.

The Yale Library has, for close to three decades, been cognizant of and aggressive about providing access to the numerous emerging forms of scholarly and popular publications appearing in "new" electronic formats, in particular those delivered via the Internet and increasingly through a Web interface. Initially, indexing and abstracting services and other "reference" type works found their way into electronic formats and rapidly displaced their former print instantiations. By 1996, full-text content from serious and substantive players such as Academic Press (AP) and JSTOR were entering the marketplace, and libraries -- and their readers -- became quickly captivated by the utility, effectiveness, efficiency, and convenience of academic content freed from its traditional fixed formats. By the summer of 2000, Yale Library, like most of its peer institutions, was spending over $1 million annually on these new publication forms, offering several hundred reference databases and thousands of full-text electronic journals to its readers. Expenditures for electronic content and access paid to outside providers last academic year alone totaled nearly $1.8 million. In addition, we spend increasing sums both on the creation of digital content and on the tools to manage digital content internally -- e.g., a growing digitized image collection, a fledgling collection of university electronic records, digital finding aids for analog materials in our collections, and our online library management system, which includes but is not limited to the public access catalog.

The electronic resources that Yale makes available to readers, both on its own and in conjunction with its partners in the NorthEast Research Libraries consortium (NERL), quickly become heavily used and wildly popular -- and increasingly duplicative, in terms of effort and price, with a number of print reference works, journals, and books. The Yale Library, with its significant resources, 300 years of collections, and long-term commitment to acquiring and preserving collections not only for its own but also for a global body of readers, has acted cautiously and prudently in retaining print texts not only for immediate use but also for long-time ownership and access. It has treated its growing, largely licensed (and thus un-owned) electronic collections, moreover, as a boon for reader productivity enhancement and convenience, even to the point of acquiring materials that seem to duplicate content but offer different functionality.

Nonetheless, it is clear that the Library (and this is true of all libraries) cannot endlessly continue on such a dual pathway, for several compelling reasons: 1) the growing level of print and electronic duplication is very costly in staff resources and to collections budgets; 2) increasingly, readers, at least in certain fields such as sciences, technology, and medicine (STM), as well as certain social sciences, strongly prefer the electronic format and the use of print in those areas is rapidly diminishing; and 3) traditional library stacks are costly to build or renovate. All libraries face what NERL members have dubbed the "Carol Fleishauer"[3] problem: "We have to subscribe to the electronic resources for many good reasons, but we cannot drop print -- where we might wish to -- because we do not trust the permanence of electronic media." The challenge for us all, then, is how to solve the problem that Carol so pointedly articulated at a NERL meeting.

When the Andrew W. Mellon Foundation first began to signal its keen interest in long-term preservation of digital resources, the Yale Library had already acquired, loaded on site, and migrated certain selected databases and full-text files for its readers. The Library had also explored, between 1997 and 1999, the possibility of serving as a local repository site for either all of Elsevier Science's more than 1,100 full-text e-journal titles or at least for the 300 or so that had been identified by NERL members as the most important for their programs. Yale information technology (IT) staff experimented with a year's worth of Elsevier electronic journals, using the Science Server software for loading and providing functionality to those journals. In the end, the fuller functionality of Science Direct (Elsevier's commercial Web product) proved more attractive for immediate access, particularly as interlinking capabilities between publishers were developed and were more easily exploitable from the commercial site than through a local load. Nonetheless, Yale's positive experience in working with Elsevier content led our staff enthusiastically to consider Elsevier as a possible e-journal archive planning partner.

When we were invited to consider applying for a planning grant (summer 2000) we contacted Karen Hunter, Elsevier Science's Senior Vice President for Strategy, who signaled a keen personal and corporate interest in long-term electronic archiving solutions. Additional attractions for Yale in working with Elsevier related to the huge amount of important content, particularly in STM, that this publisher provides. It also seemed to us that the strong commercial motivation of a for-profit publisher made such a partnership more important and interesting, at least initially, than one with a not-for-profit publisher. That is, a for-profit entity might not feel so much of a long-term or eternal commitment to its content as would a learned society. We have learned in the course of this planning project that Elsevier has its own serious commitment in this area. With Ms. Hunter and other senior Elsevier staff, a self-identified Yale Library team began to formulate e-journal questions and opportunities, and Elsevier officers strongly supported our submission of the Fall 2000 application to Mellon.

Throughout the project, regular meetings between Yale and Elsevier team members have been held, and key topics have been identified and pursued therein, as well as by phone, e-mail, through the work of small sub-groups, through site visits by team members and visitors to our establishments. The principal lines of inquiry have been in the following areas: 1) "trigger" events; 2) a beginning exploration of the fascinating area of "archival uses," i.e., how real-life users might use the kind of archive we were proposing to develop; 3) contractual and licensing issues; 4) metadata identification and analysis, particularly through comparison and cross-mapping with datasets recommended by such players as the British Library and OCLC/RLG; and 5) technical issues, beginning with an examination of Elsevier's production and workflow processes, and with particular emphasis after summer of 2001 on building a small prototype archive based on the OAIS (Open Archival Information Systems) model,[4] focusing on the archival store component. Interlaced have been the economic and sustainability questions that are crucial to all electronic preservation projects.

In July 2001, the acquisition by Elsevier Science of a number of Harcourt properties and their component journal imprints was announced. This real-life business event presented interesting and significant challenges, not only to Elsevier Science but also prospectively to the Yale Electronic Archive (YEA). At that time, Elsevier had not fully formulated its organizational or technical plans for the new imprints, which included not only AP (whose IDEAL service represented one of the earliest commercially-licensed groups of important scientific journals, introduced to the marketplace in 1995), but also Harcourt, Saunders, Mosby, and Churchill Livingstone, i.e., primarily medical titles which are to be integrated into a new Elsevier division called Health Science. Elsevier staff believe that the imprints of the acquired titles will survive; it is not clear whether the IDEAL electronic platform will continue or whether (more likely) it will be integrated into Science Direct production processes.

The assumptions listed above speak to four major activities of the YEA, which are 1) preservation planning, 2) administration of the archive, 3) access to the archive, and 4) ingestion of content. Like other Mellon research projects, the YEA defines activities associated with these processes within the context of the OAIS reference model of a digital archive. A fortiori, the model also states that implementations will vary depending upon the needs of an archival community.

Research conducted during the planning year has identified four different approaches to preservation: emulation, migration, hard copy, and computer museums. These different approaches should not, however, be viewed as mutually exclusive. They can be used in conjunction with each other. Additionally, as we are not pursuing the present study as simply an academic exercise but rather, as a very practical investigation of what it will take to build, operate, and maintain a fully functioning production-mode digital archive, we cannot possibly discount the financial implications of the different approaches and the impact of standards on a choice of approach.

In choosing our approach, we quickly discounted both hard copy and computer museums. Hard copy decays over time and multimedia objects cannot be printed. Computer museums were also discounted as impractical. To function as an archive, computer museum equipment would need to be operational, not simply a collection of static exhibits. In turn, operation would lead to inevitable wear and tear on the equipment, with the consequential need for maintenance and repair work. When considering the repair of "antique" computer equipment, one has to ask about the source of spare parts -- do all of them have to be hand-made at enormous expense? Even if money were available for such expensive work, does the "antique" equipment come with adequate diagnostic and testing equipment, wiring diagrams, component specifications, and the like, which would make the "museum" choice technically feasible in the first place? Even if the antique equipment were to receive only the most minimal usage, the silicon chips would deteriorate over time. In such a museum environment, one would question whether we would be in danger of losing our focus, ending up as a living history-of-computers museum rather than an archive of digital materials.

Rejecting hardcopy and museum options left the team with two very different approaches to the storage of content in an OAIS archive. One approach to content preservation is to store objects based upon emerging standards such as XML and then migrate them to new formats as new paradigms emerge. The other approach, advanced by and associated with one of its chief proponents, Marc Rothenberg, is to preserve content through emulation. Both approaches have potential and value, but probably to different subcultures in the archival community. A goal of standards is to preserve the essential meaning or argument contained in the digital object. Archives that are charged with the responsibility of preserving text-based objects such as e-journals are likely to adopt a migratory approach. Archives that need to preserve an exact replication or clone of the digital objects may choose, in the future, to deploy emulation as an archival approach. Both approaches are rooted in the use of standards. Contrary to an argument advanced in a research paper by Rothenberg,[5] YEA team members maintain that standards, despite their flaws, represent an essential component of any coherent preservation strategy adopted.

That said, Rothenberg criticized the use of migration as an approach to digital longevity. Rothenberg does make an insightful and enterprising case for the practice of emulation as the "true" answer to the digital longevity problem. The "central idea of the approach, " he writes, "is to enable the emulation of obsolete systems on future, unknown systems, so that a digital document's original software can be run in the future despite being obsolete." Rothenberg avers that only by preservation of a digital object's context -- or, simply stated, an object's original hardware and software environment -- can the object's originality (look, feel, and meaning) be protected and preserved from technological decay and software dependency.

The foundation of this approach rests on hardware emulation, which is a common practice in the field of data processing. Rothenberg logically argues that once a hardware system is emulated, all else just naturally follows. The operating system designed to run on the hardware works and software application(s) that were written for the operating system also work. Consequently, the digital object behaves and interacts with the software as originally designed.

However, emulation cannot escape standards. Processors and peripherals are designed with the use of standards. If the manufacturer of a piece of hardware did not adhere 100 percent to the standard, then the emulation will reflect that imperfection or flaw. Consequently, there is never a true solution, as suggested by Rothenberg, that a generalized specification for an emulator of a hardware platform can be constructed. In the data processing trenches, system programmers are well acquainted with the imperfections and problems of emulation. For example, the IBM operating system MVS never ran without problems under IBM's VM operating system. It was a good emulation but it was not perfect. Another major problem with emulation in a practical sense is its financial implications. The specification, development, and testing of an emulator require large amounts of very sophisticated and expensive resources.

At this stage, the YEA team believes the most productive line of research is a migratory approach based upon standards. Standards development must, therefore, feature front and center in the next phase of e-journal archiving activities. If one listens closely to academic discourse, the most seductive adverb of all is one not found in a dictionary; it is spelled "just" and pronounced "jist" and is heard repeatedly in optimistic and transparent schemes for making the world a better place. If scientists would "jist" insist on contributing to publishing venues with the appropriate high-minded standards of broad access, we would all be better off. If users would "jist" insist on using open source operating systems like Linux, we would all be better off. If libraries would "jist" spend more money on acquisitions, we would all be better off.

Many of those propositions are undoubtedly true, but the adverb is their Achilles' heel. In each case the "jist" masks the crucial point of difficulty, the sticking point to movement. To identify those sticking points reliably is the first step to progress in any realistic plan for action. In some cases, the plan for action itself is arguably a good one, but building the consensus and the commonality is the difficulty; in other cases, the plan of action is fatally flawed because the "jist" masks not merely a difficulty but an impossibility.

It would be a comparatively easy thing to design, for any given journal and any given publisher, a reliable system of digital information architecture and a plan for preservation that would be absolutely bulletproof -- as long as the other players in the system would "jist" accept the self-evident virtue of the system proposed. Unfortunately, the acceptance of self-evident virtue is a practice far less widely emulated than one could wish.

It is fundamental to the intention of a project such as the YEA that the product -- the preserved artifact -- be as independent of mischance and the need for special supervising providence as possible. That means that, like it or not, YEA and all other seriously aspiring archives must work in an environment of hardware, software, and information architecture that is as collaboratively developed and as broadly supported as possible, as open and inviting to other participants as possible, and as likely to have a clear migration forward into the future as possible.

The lesson is simple: standards mean durability. Adhering to commonly and widely recognized data standards will create records in a form that lends itself to adaptation as technologies change. Best of all is to identify standards that are in the course of emerging, i.e., that appear powerful at the present moment and are likely to have a strong future in front of them. Identifying those standards has an element of risk about it if we choose the version that has no future, but at the moment some of the choices seem fairly clear.

Standards point not only to the future but also to the present in another way. The well-chosen standard positions itself at a crossroads, from which multiple paths of data transformation radiate. The right standards are the ones that allow transformation into as many forms as the present and foreseeable user could wish. Thus PDF is a less desirable, though widely used, standard because it does not convert into structured text. The XML suite of technology standards is most desirable because it is portable, extensible, and transformative: it can generate everything from ASCII to HTML to PDF and beyond.

The team discovered during the course of its explorations that there is no single type of archive. While it is true that all or most digital archives might share a common mission, i.e., the provision of permanent access to content, as we noted in our original proposal to the Mellon Foundation, "This simple truth grows immensely complicated when one acknowledges that such access is also the basis of the publishers' business and that, in the digital arena (unlike the print arena), the archival agent owns nothing that it may preserve and cannot control the terms on which access to preserved information is provided."

In beginning to think about triggers, business models, and sustainability, the project team modeled three kinds of archival agents. The first two types of archives include a de facto archival agent, defined as a library or consortium having a current license to load all of a publisher's journals locally, or a self-designated archival agent. Both of these types are commercial transactions, even though they do not conduct their business in the same ways or necessarily to meet the same missions. The third type of archive is a publisher-archival agent partnership and the focus of our investigation. Whether this type can now be brought into existence turns on the business viability of an archive that is not heavily accessed. Project participants varied in their views about whether an archive with an as yet uncertain mission can be created and sustained over time and whether, if created, an individual library such as Yale or a wide-reaching library enterprise like OCLC would be the more likely archival partner.

So when does one access the archive? Or does one ever access it? If the archive is never to be accessed (until, say, material passes into the public domain, which currently in the United States is seventy years plus the lifetime of the author or rights holder), then the incentives for building it diminish greatly, or at least the cost per use becomes infinite. There is talk these days of "dark" archives, that is, collections of data intended for no use but only for preservation in the abstract. Such a "dark" archive concept is at the least risky and in the end possibly absurd.

Planning for access to the e-archive requires two elements. The less clearly defined at the present is the technical manner of opening and reading the archive, for this will depend on the state of technology at the point of need. The more clearly defined, however, will be what we have chosen to call "trigger" events. In developing an archival arrangement with a publisher or other rights holder, it will be necessary for the archive to specify the circumstances in which 1) the move to the archive will be authorized, which is much easier to agree to than the point at which 2) users may access the archive's content. The publisher or rights holder will naturally discourage too early or too easy authorization, for then the archive would begin to attract traffic that should go by rights to the commercial source. Many rights holders will also naturally resist thinking about the eventuality in which they are involuntarily removed from the scene by corporate transformation or other misadventure, but it is precisely such circumstances that need to be most carefully defined.

Project participants worked and thought hard to identify conditions that could prompt a transfer of access responsibilities from the publisher to the archival agent. These conditions would be the key factors on which a business plan for a digital archive would turn. The investigation began by trying to identify events that would trigger such a transfer, but it concluded that most such events led back to questions about the marketplace for and the life cycle of electronic information that were as yet impossible to answer. Team members agreed that too little is known about the relatively young business of electronic publishing to enable us now to identify definitively situations in which it would be reasonable for publishers to transfer access responsibility to an archival agent.

That said, some of the possible trigger events identified during numerous discussions by the project team were:

Long-term physical damage to the primary source. Note that we have not imagined the e-journal archive to serve as a temporary emergency service. We expect formal publishers to make provision for such access. Nevertheless, in the case of cataclysmic event, the publisher could have an agreement with the archive that would allow the publisher to recopy material for ongoing use.

Loss of access or abdication of responsibility for access by the rights holder or his/her successor, or no successor for the rights holder is identified. In other words, the content of the archive could be made widely available by the archive if the content is no longer commercially available from the publisher or future owner of that content. We should note that at this point in time, we were not easily able to imagine a situation in which the owner or successor would not make provision precisely because in the event of a sale or bankruptcy, content is a primary transactional asset. But that is not to say that such situations will not occur or that the new owner might not choose to deal with the archive as, in some way, the distributor of the previous owner's content.

Lapse of a specified period of time. That is, it could be negotiated in advance that the archive would become the primary source after a negotiated period or "moving wall," of the sort that JSTOR has introduced into the e-journal world's common parlance. It may be that the "free science" movement embodied in PubMed Central or Public Library of Science might set new norms in which scientific content is made widely available from any and all sources after a period of time, to be chosen by the rights owner. This is a variant on the "moving wall" model.

On-site visitors. Elsevier, our partner in this planning venture, has agreed that at the least, its content could be made available to any onsite visitors at the archive site, and possibly to other institutions licensing the content from the publisher. Another possibility is provision of access to institutions that have previously licensed the content. This latter option goes directly to development of financial and sustainability models that will be key in Phase II.

Archival Uses. Elsevier is very interested in continuing to explore the notion of so-called "archival uses" which represent uses very different to uses made by current subscribers in support of today's active science. Elsevier has stated that if we can identify such "archival uses," it might be willing to consider opening the archive to those. Some such uses might be studies in the history, sociology, or culture of sciences, for example. This thread in our planning processes has motivated the YEA team to devote some time to early exploration of archival uses with a view to expanding and deepening such exploration in Phase II.

Metadata Uses. In the course of preservation activity it could be imagined that new metadata elements and structures would be created that would turn out to have use beyond the archive. Appropriate uses of such data would need to be negotiated with the original rights holder.

The task of archiving electronic journals may be divided into five parts: the difficult part (infrastructure development and startup), the easier part (maintenance), the sometimes tricky part (collaborations and standards), the messy part (comprehensiveness), and the part where it becomes difficult again (new technologies, migration).

Needless to say, the above overview is somewhat simplified. For example, in our planning year, we were surprised to find just how few of the 1,100+ Elsevier e-journal titles carried complex information objects, compared to what we expected to find. Complex media, data sets, and other electronic-only features exist that have yet to find their place as regular or dominant players in e-journals, and creating ways to deal with these types of digital information -- let alone standard ways -- will be costly, as are all initial structural activities (see #1 above).

That said, it appears that willing collaborators have yet a little time both to address and to solve the hefty problems of presenting and archiving complex digital information objects. To archive a single e-journal or small set of journals is to do relatively little. But to develop standards that will serve e-preservation well -- let alone to facilitate access to the most simple of e-archives that begin to bloom like a hundred flowers -- all the players will need to work together. We imagine an aggregation of archiving efforts, whether in physical co-location or at least virtual association and coordination.

But how might such archival universes be organized?

Each of these alternate universes has its own gravitational force and all will probably come into existence in one form or another. Such multiplicity creates potentially severe problems of scalability and cost. One remedy could be for official archives to operate as service providers feeding other archives. Hence, a publisher's agreed archive could feed some of its journals to one subject-based archive and others to national archives.

One way to begin to anticipate and plan for this likely multiplicity would be to create a consortium now of interested parties to address the difficult issues such as redundancy, certification, economic models, collection of fees, standards, and so on. No one organization can solve these problems alone, but coordination among problem-solvers now and soon will be very cost-effective in the long run. In OCLC's proposal to create a digital preservation cooperative,[7] and, on a larger scale in the Library of Congress's recent National Digital Information Infrastructure Preservation Program,[8] we may be seeing the emergence of such movements. It may be possible to turn the Mellon planning projects into such an overarching group of groups.

No preservation ambitions will be realized without a sustainable economic model. As we have noted above, the costs of archiving are much in dispute and our study will examine those costs in great detail in the next phase. For now, it would appear that the initial costs are high, although manageable, and the ongoing costs, at least for standard publisher's journals, could be relatively predictable and eventually stable over time.

If that is true, then various models for paying for the archiving process suggest themselves. This is an area about which there has been much soft discourse but in which there has been little experience, save perhaps for JSTOR whose staff have given the topic a great deal of thought.

Up-front payment. The most dramatic and simple way to finance the e-journal archives would be the "lifetime annuity model": that is, users (presumably institutional entities, such as libraries, professional societies, governments, or cultural institutions, but some speak of enhanced "page charges" from authors or other variants on current practices) pay for a defined quantum of storage and with that one-time payment comes an eternity of preservation. The up front payment would be invested partly in ongoing archival development and partly in an "endowment" or rainy day fund. The risk in this case is that inadequate funding may lead to future difficulties of operation.

Ongoing archival fees. An "insurance premium" on the other hand could give an ongoing supply of money, adjustable as costs change, and modest at all stages. This reduces the risk to the provider but increases the uncertainty for the beneficiary. The ongoing fee could be a visible part of a subscription fee or a fee for services charged by the archive.

The traditional library model. The library (or museum or archive) picks up the tab and is funded by third-party sources.

Fee for services operation. The archive provides certain services (special metadata, support for specialized archives) in return for payments.

Hybrid. If no single arrangement seems sufficient -- as it likely will not -- then a hybrid system likely will emerge, perhaps with one set of stakeholders sharing the up-front costs while another enters into agreement to provide ongoing funding for maintenance and potential access.

Much more could be said on the topic of who pays but at the moment most of it would be speculation. The choice of models will influence development of methods for paying fees and the agents who will collect those fees. Before making specific recommendations it will be important for our project to develop a much more specific sense of real costs of the e-archive. We imagine that we might want to develop both cost and charging models in conjunction with other libraries, i.e., prospective users of the archive. In Yale's case the collaborative effort might happen with our local electronic resource licensing consortium NERL.

YEA and Elsevier Science have come to basic agreement on what they would be comfortable with as a model license. In some areas alternatives are clearly available and other archival agencies working with other publishers will choose different alternatives. Reaching a general agreement was, however, surprisingly easy as the agreement flowed naturally out of the year-long discussions on what we were trying to accomplish. The current draft license is not supplied in this document because it has a number of "unpolished" areas and some unresolved details, but it could be submitted and discussed upon request.

The team made certain choices with regard to the contractual issues noted above:

The model license is still evolving and not yet ready for signature. However, there are no identified points of contention -- only points for further reflection and agreement on wording. All the participants were very much pleased with the team's ability to come to early understandings of licensing issues and to resolve some of these at the planning stage. This success arises out of close working relationships and communications over about a year-and-a-half of cooperative effort.

The first area in which there might be useful approaches is that of archival appraisal, i.e., the selection of those materials worth the resources needed for their long-term preservation and ongoing access. Archival appraisal considers the continuing need of the creating entity for documentation in order to carry out its mission and functions and to maintain its legal and administrative accountability, as well as other potential uses for the materials. These other uses generally fall into the category of support for historical research, although there may be others such as establishing and proving the existence of personal rights which may also be secondary to the original purpose of the documentation in question.

Archivists also consider the context of the documentation as well as its content in determining long-term significance. In some cases, the significance of the documentation lies in the particular content that is recorded; the presentation of that content is not critical to its usefulness or interpretation. The content of the documentation can be extracted, put into other applications, and made to serve useful purposes even as it is divorced from its original recording technology and form. In other cases, however, the role of documentation as evidence requires that the original form of the document and information about the circumstances under which it was created and used also be preserved in order to establish and maintain its authenticity and usefulness.

With these selection approaches in mind, a number of issues arose in the e-journal archiving context and in the work of the team. The first question was whether it was sufficient for the archive to preserve and provide access to "just" the content of the published material -- primarily text and figures -- in a standard format, which might or might not be the format in which the publisher distributed the content. Preserving only the content, insofar as that is possible, foregoes the preservation of any functionality that controlled and facilitated the discovery, navigation, and presentation of the materials on the assumption that functionality was of little or no long-term research interest. The decision to preserve content only would eliminate the need to deal with changing display formats, search mechanisms and indices, and linking capabilities.

While the group has adopted this narrow definition of the scope of the archive as a working assumption, such a narrow approach does preclude the study of the diplomatics of these documents -- "digital paleography," as one of our advisors termed it. How essential to future researchers' interpretations of the use of these documents is it for them to know what tools contemporary users had available to them, e.g., indices that did not address particular components of the document, thus making them unfindable through the publisher's interface? At the conclusion of the planning period the team had not changed the main focus of its attention on content, but it was sufficiently intrigued by the issues of digital paleography that it will propose that this assumption be investigated more thoroughly in its implementation proposal.

The long-held approach in the archival profession governing how archives are organized, described, and provided to users once they become part of the repository's holdings is deeply informed by the principle of provenance and the rules that flow from it: respect des fonds (records of a creator should remain together) and original order (which has significance for the interpretation of records and should be preserved whenever possible). These principles reflect the nature of archival records. They are by-products created by an organizational entity in the course of carrying out its functions. The primary significance of the records is as evidence of those functions and activities. These principles reflect the needs of research for bodies of materials that are as strongly evidential as possible and reflect minimal interaction by custodial agencies other than the creator. The assumption is that solid historical research and interpretation require knowledge of the circumstances under which the materials were created and maintained and not just access to the raw content.

Access to archival materials is often characterized by two factors that take advantage of the provenance approach. Searches are often conducted to document a particular event or issue rather than for a known item; they may also be based on characteristics of the creators rather than on characteristics of the records themselves. Comprehensive and accurate recording of the circumstances of creation, including characteristics of records creators and the relationships among them, are central parts of archival description. The implications for developing an approach to downstream uses of e-journal literature include the potential need of contextual metadata regarding the authors and other circumstances affecting the publication of a given article/issue that are not found in a structured way in the published materials. Information regarding the context in which the article was submitted, reviewed, and edited for publication is important in studies of scholarly communication, especially as to questions of how institutional affiliations might be important in certain lines of inquiry and who had the power to accept or reject submissions.

Some of this information is explicitly disseminated in online products, e.g., in the form of members of an editorial board or descriptions of the purpose and audience of the journal, but it may be presented separately from any particular volume, issue, or article; may reflect only current (and not historic) information; and is rarely structured or encoded in such a way as to facilitate its direct use in scholarly studies. Other information about the context of creation and use that historians of science might find useful is not published; rather, it is found in the publisher's records of the review process and circulation figures. Capturing and linking of title-level publication information are additional areas of investigation that the team intends to pursue in its implementation proposal.

The mass of archival records that repositories select for long-range retention and are responsible for, and the imperative of the principle of provenance to maintain and document the recordkeeping system in which the records were created and lived, combine to foster the archival practice of top-down, hierarchical, and collective description. This type of descriptive practice provides both a way of reflecting the arrangement of the original recordkeeping system and of allowing the archival agency to select for each body of records the level beyond which the costs of description outweigh the benefits of access, and completing its descriptive work just before that point is achieved.

This principle and practice highlight for scientific journals the importance of preserving the relationship among the materials that the publisher was distributing, especially the need to link articles that the publisher presented as a "volume," "special issue," or some other sort of chronological or topical grouping. These relationships represent another form of contextual information important to the study of scholarly communications, in terms of which articles were released simultaneously or in some other relationship to each other. While the team recognized the need to be aware of new forms of publishing that would not necessarily follow the traditional patterns adopted by the hard-copy print world, it asserted that those structures do need to be saved as long as they are used.

With respect to other methods of navigating among digitally presented articles, such as linking to articles cited, the team found that many of these capabilities existed not as part of the content, but as added functionality that might be managed by processes external to the content or to the publisher's product (e.g. CrossRef). The team felt that these capabilities should be preserved as part of the archive, necessitating the need to maintain an enduring naming scheme for unambiguous identification of particular pieces. The plan for the implementation project will include a closer look at the requirements for supporting important navigational capabilities.

Finally, the authenticity of any document that purports to be evidence rests in some part on a chain of custody that ensures that the document was created as described and that it has not been altered from its original form or content. Once an archival agency takes charge of documentation it is obligated to keep explicit records documenting the circumstances of its transfer or acquisition and any subsequent uses of it. Records are rarely removed, either for use or retrospective retention by an office, but when this is necessary the circumstances of that action need to be documented and available. This assumption, along with the unique nature and intrinsic value of the materials, leads to the circumstance of secure reading rooms for archival materials and all of the security paraphernalia associated with them, as well as to detailed recordkeeping of use and work performed on the records.

The assumption that the archival agency is responsible for preserving the "authentic" version of documentation suggests that transfer of content to the official archival agency should take place as soon as the publisher disseminates such content, and that once placed into the archive content will not be modified in any way. This includes instances of typographical errors, the release of inaccurate (and potentially dangerous) information, or the publication of materials not meeting professional standards for review, citation, and similar issues. Instead, the archive should maintain a system of errata and appropriate flagging and sequestering of such materials that were released and later corrected or withdrawn, ensuring that the record of what was distributed to the scholarly community, however flawed, would be preserved.

Issues related to authenticity also suggest that one circumstance under which transferred content could be released, even while the publisher retains a business interest in it, is when questions are raised as to the authenticity of content still available under normal business arrangements. Longer-term safeguards will need to be in place within the archival repository to ensure the authenticity of the content.

Other issues relating to the nature and mission of an archival repository appear elsewhere in this report, especially in the discussion of trigger events. The issues discussed in this section, however, are especially germane to the question of how anticipated use of preserved electronic journals should inform the selection of materials. The Yale-Elsevier team has found many archival use topics central to the definition and purpose of an archive for electronic journals and plans to pursue them more completely in the implementation project.

It is impossible to create a system designed to authenticate, preserve, and make available electronic journals for an extended period of time without addressing the topic of metadata. "Metadata" is a term that has been used so often in different contexts that it has become somewhat imprecise in meaning. Therefore, it is probably wise to begin a discussion of metadata for an archival system by narrowing the array of possible connotations. In the context of this investigation, metadata makes possible certain key functions:

Taken together, both types of metadata facilitate the preservation of the content for the future (preservation metadata). Preservation ensures the retrievability of protected materials, their authentication, and their content.

Using metadata to describe the characteristics of an archived item is important for a number of reasons. With care, metadata can highlight the sensitivity to technological obsolescence of content under the care of an archival agency (i.e., items of a complex technical nature that are more susceptible to small changes in formats or browsers.) Metadata can also prevent contractual conflicts by pinpointing issues related to an archived item's governance while under the care of an archive; e.g., "the archive has permission to copy this item for a subscriber but not for a nonsubscriber." Finally, metadata can permit the archival agency to examine the requirements of the item during its life cycle within the archive; e.g., "this object has been migrated four times since it was deposited and it is now difficult to find a browser for its current format."[9]

The Open Archival Information System (OAIS) model to which the YEA project has chosen to conform refers to metadata as preservation description information (PDI). There are four types of PDI within OAIS: 1) reference information, 2) context information, 3) provenance information, and 4) fixity information. Not all of these forms of PDI need be present in the Submission Information Package (SIP) ingested by the archive, but they all must be a part of the Archival Information Package (AIP) stored in the archive. This implies that some of these PDI elements are created during ingestion or input by the archive.

Reference Information refers to standards used to define identifiers of the content. While YEA uses reference information and supplies this context in appendices to our metadata element set, we do not refer to it as metadata. Context Information documents the relationships of the content to its environment. For YEA, this is part of the descriptive metadata. Provenance Information documents the history of the content including its storage, handling, and migration. Fixity Information documents the authentication mechanisms and provides authentication keys to ensure that the content object has not been altered in an undocumented manner. Both Provenance and Fixity are part of administrative metadata for YEA.

Given the focus YEA has chosen to place on a preservation model that serves as an archive as well as a guarantor for the content placed in its care, authenticity was an issue of importance for the group to explore. In its early investigations, the team was much struck by the detailed analysis of the InterPARES project on the subject of authenticity. While some of the InterPARES work is highly specific to records and manuscripts -- and thus irrelevant to the journal archiving on which YEA is focusing -- some general principles remain the same. It is important to record as much detail as possible about the original object brought under the care of the archive in order both to prove that a migrated or "refreshed" item is still the derivative of the original and to permit an analysis to be conducted in the future about when and how specific types of recorded information have changed or are being reinvented, or where totally new forms are emerging.[10]

Finally, as YEA has examined the issue of metadata for a system designed to authenticate, preserve, and make available electronic journals for an extended length of time, we have tried to keep in mind that metadata will not just be static; rather, metadata will be interacted with, often by individuals who are seeking knowledge. To this end, we acknowledge the four issues identified by the International Federation of Library Associations and Institutions (IFLA) in the report on functional requirements for bibliographic records: metadata exist because individuals want to find, identify, select, or obtain informational materials.[11]

YEA began its analysis of needed metadata for a preservation archive of electronic journals by conducting a review of extant literature and projects. In this process the team discovered and closely explored a number of models and schemes. The first document -- and the one we returned to most strongly in the end -- described the OAIS model, although OAIS provides only a general framework and leaves the details to be defined by implementers.[12] We also examined the Making of America's testbed project white paper[13] and determined it was compatible with OAIS. Next, we examined the 15 January 2001 RLG/OCLC Preservation Metadata Review document[14] and determined that while all of the major projects described (CEDARS, NEDLIB, PANDORA) were compliant with the OAIS structure, none of them had the level of detail, particularly in contextual information, that we believed necessary for a long-term electronic journal archive. We also explored the InterPARES project (mentioned above) and found there a level of detail in contextual information that we had not seen delineated in the RLG/OCLC review of the other projects.

At the same time, the library and publisher participants in the project were exploring the extant metadata sets used by Elsevier Science to transport descriptions of their journal materials for their own document handling systems and customer interfaces. In addition to their EFFECT standard (see section describing Elsevier Science's Technical Systems and Processes), we also examined portions of the more detailed Elsevier Science "Full Length Article DTD 4.2.0."[15] Due to the solid pre-existing work by Elsevier Science in this area and the thorough documentation of the metadata elements that Elsevier Science is already using, we were able to proceed directly to an analysis of the extant Elsevier metadata to determine what additional information might need to be created or recorded during production for and by YEA.

About halfway through the project year, the team made connections with British Library staff who were themselves just completing a metadata element set definition project and who generously shared with the team their draft version. While the British Library draft document was more expansive in scope than the needs of the YEA project (i.e., the British Library document covers manuscripts, films, and many other items beyond the scope of any e-journal focus), the metadata elements defined therein and the level of detail in each area of coverage were almost precisely on target for what the e-archiving team sought to create. Thus, with the kind consent of contacts at the British Library, the team began working with the draft, stripping away unneeded elements, and inserting some missing items.

In the fall of 2001, the YEA team committed to creating a working prototype or proof-of-concept which demonstrated it would indeed be possible to ingest data supplied by Elsevier Science into a minimalistic environment conducive to archival maintenance. The prototype-building activity briefly diverted the metadata focus from assembling a full set of needed elements for the archival system to defining a very minimal set of elements for use in the prototype. The technical explorations of the prototype eventually led us to simply use the metadata supplied by Elsevier and the prototype metadata element set was never used. The one remaining activity associated with metadata performed for the prototype was to map the Elsevier EFFECT metadata to Dublin Core so that it could be exposed for harvesting.

Once the prototype subset element set was identified, YEA returned to the question of a complete metadata element set for a working archive. As the British Library draft document was examined, reviewed, and assessed, many decisions were made to include or exclude elements, particularly descriptive metadata elements. These decisions were informed in part by the recurring theme of whether the presence of such an item of information would assist individuals performing inquiries of the archive. The questions related to uses of scholarly journal materials for archival explorations are dealt with more fully elsewhere in this report.

The full metadata element set was completed by YEA as a recommended set of metadata to be used in a future full archive construction. It is important to reiterate that our approach to producing this set of metadata was inclusive. In creating an archival architecture it is not enough to delineate the descriptive metadata that must be acquired from the publisher or created by the archive while leaving out the administrative metadata elements that permit the archive to function in its preserving role. Neither is it sufficient to focus on the administrative metadata aspects that are unique to an archive while setting aside the descriptive metadata elements, i.e., assuming they are sufficiently defined by other standards. Preservation metadata are the conflation of the two types of metadata and, in fact, both types of metadata work jointly to ensure the preservation of and continuing access to the materials under the care of the archive.

One other fact may be of interest to those reviewing the description of metadata elements for the YEA: where possible, we used external standards and lists as materials upon which the archive would depend. For example, we refer to the DCMI-Type Vocabulary[16] as the reference list of the element called "resource type."

We certainly do not expect that the element set created by YEA will proceed into implementation in a future full archive construction without any further changes. It will undoubtedly be influenced by work done by other groups such as the E-Journal Archive DTD Feasibility Study[17] prepared for the Harvard University Library e-journal archiving project. However, we now have a reference by which to assess whether proposed inclusions, exclusions, or modifications fit the structure we imagine an archive will need properly to preserve electronic journals.

In the next phase of the e-archiving project, the YEA desires further to define and refine metadata needed for a system designed to authenticate, preserve, and make available electronic journals for an extended period of time. We will need to connect with others working informally or formally to create a standard or standards for preservation metadata. As noted above, further investigations may influence a revision to the initial metadata set defined during the planning phase. Additionally, we intend to rework the element set into an XML schema for Open Archives Initiative (OAI) manifestation and harvesting. With our small prototype, we have demonstrated that OAI harvesting can occur from the simple Dublin Core metadata set to which we mapped the Elsevier EFFECT elements. However, OAI interaction can occur much more richly if a fuller dataset is in use, and we intend to accomplish this schema transformation to enable fuller interaction with the archive as it develops.[18]

As the next phase moves forward, another avenue of exploration will be to assess and examine our metadata element choices in a live environment. We are most particularly interested in testing those elements included or excluded on the basis of assumptions made regarding the likelihood of archival inquiries targeting specific elements for exploration. Such choices can only be validated over time and with the interaction of individuals conducting authentic research into the history of their fields. Finally, we look forward to testing our element choices for administrative metadata under the stress of daily archive administration and maintenance. Only in such a live environment can an archive be truly confirmed as a functioning entity for preserving the materials under its care.

Elsevier Science is a major producer of scholarly communication and scientific journals that are distributed globally. The headquarters for production, along with the electronic warehouse, is located in Amsterdam, Netherlands. There the company maintains two office buildings and deploys several hundred staff to organize, produce, and distribute its content. The production of electronic scholarly information is a highly complex process that occurs in a distributed geographical environment involving many businesses beyond Elsevier Science. Changes to the manufacturing process can take years to percolate through the entire chain of assembly and are considered significant business risks. Consequently, Elsevier is moved to make changes to production only when compelling market demands exist. For example, the Computer Aided Production (CAP) workflow is now under modification because Science Direct, an internal customer of Elsevier Science, is experiencing market pressure to bring published items to its customers in a shorter time than ever before.

Prior to the creation of the Electronic Warehouse (EW) in 1995, Elsevier Science had no standard processes to create and distribute journals or content. The production of journals was based upon a loose confederation of many smaller publishing houses owned by Elsevier Science. Content was produced using methods that were extant when Elsevier acquired a given publisher. Consequently, prior to the creation of the EW there was no uniformity in the structure or style of content marketed under the name of Elsevier Science. Each publishing house set its own standards for creation and distribution. The lack of a central infrastructure for creating and distributing content also served as an impediment to the rapid distribution of scholarly communication to the market.

With the creation of networks in the early 1990s the perception of time delay amplified. Scientists began to use the network themselves to share communications with one another instantly. The scholarly community would no longer accept long delays between the submission of manuscripts to a publisher and their appearance in paper journals. Scientists and publishers realized that reporting research in electronic format could significantly close the time gap between publication and distribution of content to the scholarly community. The origin of Elsevier Science's Electronic Warehouse is rooted in this realization. Elsevier Science's early solution to the problem was to support a research project known as The University Licensing Program commonly referred to as TULIP.[19]

The TULIP Project (1992-1996) grew out of a series of Coalition for Networked Information (CNI) meetings in which Elsevier Science, a CNI member, invited academic libraries to partner with it to consider how best to develop online delivery capabilities for scientific journals. The purpose of the project was to discuss the need to build large-scale systems and infrastructures to support the production and rapid delivery of such journals over a network to the scholarly community. Given a critical mass of interest from the university communities, Elsevier Science justified a large investment that would create a manufacturing function for converting paper journals into an electronic format for network distribution. This process became known as the PRECAP method for the creation of an electronic version of a journal. The creation of this conversion function served as the foundation for the present day EW. Near the end of the TULIP project plans for an EW were adopted by Elsevier Science in 1995 and built by the end of 1996. By 1997 the EW could produce over one thousand journals using a standard means of production.

The creation and success of the EW in producing and distributing journals was a very significant accomplishment for Elsevier Science because 1) many individual publishers had to be converted one by one, 2) standards for production were evolving from 1994 through 2000, and 3) suppliers who created content for the producers needed to be continuously trained and retooled to adhere to the evolving standards. At the same time, these suppliers met their obligation to produce content, on time, for Elsevier Science.

Elsevier Science maintains four production sites based in the United Kingdom (Oxford and Exeter), Ireland (Shannon), the United States (New York), and the Netherlands (Amsterdam). Each site provides content to the EW where this content is stored as an S300 dataset. The contents of each dataset represent an entire issue of a particular journal. The storage system at the EW originally used vanilla IBM technology, i.e., ADSTAR Distributed Storage Manager (ADSM), to create tape backup datasets of content stored on magnetic and optical storage. Access to the data was based only upon the file name of the S300 dataset. As of Summer 2001, the old hierarchical storage system was replaced by an all-magnetic disk-based system providing more flexibility and enabling faster throughput and production times.

The following is a concise description and discussion of the Computer Aided Production (CAP) workflow. An item is accepted for publication by means of a peer review process. After peer review the item enters the CAP workflow via the Login Function in which a publication item identifier (PII) is assigned to the content. This is a tag that the EW uses to track the item through the production process, and it also serves as a piece of metadata used for the long-term storage of the item. Since this identifier is unique it could also be used as a digital object identifier for an information package in an OAIS archive. In addition to assigning the PII, the login process also obtains other metadata about the author and item such as the first author's name, address, e-mail address, and number of pages, tables, and figures in the item. This and other similar metadata are entered into a Production Tracking System (PTS) that is maintained by the Production Control system.

The item is then sent electronically to a supplier (Elsevier has sixteen suppliers, distributed on a worldwide basis). There the item undergoes media conversion, file structuring, copy editing, and typesetting. The output of this processing is a first generation (no corrections) SGML markup of the item, a PDF file, and artwork for the item. These units of work are then sent to the author for corrections. The author makes the necessary corrections, then sends the item to Production Control where information in the PTS system is updated. Thereafter, Production Control sends the item to an Issues Manager. Any problems found in the content are worked out between the author and the Issues Manager. If there are no problems, the supplier sends the content directly to Production Control.

The Issues Manager then passes the corrections on to the supplier and begins to compile the issue. This involves making decisions about proofs, cover pages, advertising, and building of indexes. On average, an Issues Manager is responsible for five to ten journals or about fifteen thousand pages a year. Once content is received, the supplier then creates a second-generation SGML and PDF file and new artwork, if necessary. This cycle is repeated until the complete issue is assembled. Once the issue is fully assembled the Issues Manager directs the supplier to create a distribution dataset called S300 which contains the entire issue. The supplier sends this file to the EW where the file serves as input for the creation of distribution datasets for customers such as Science Direct. At EW this dataset is added to an ADSM-based storage system that serves as a depository -- not an archive -- for all electronic data produced for the EW. The S300 dataset is also sent to a printer where a paper version of the issue is created and then distributed to customers. The paper version of the journal is also stored in a warehouse. Most printing occurs in the Netherlands and the United Kingdom.

The current issue-based workflow has two serious problems. The first is that production does not produce content for distribution in a timely fashion for customers like Science Direct, and the second is that issue-based processing generates high and low periods of work for suppliers. A steady stream of work passing through the manufacturing process would be more efficient for the suppliers and would result in a more timely delivery of content to Elsevier's customers such as Science Direct. The driving force behind a need for change, as mentioned above, is not EW but rather, Science Direct as an internal customer of the EW. The resolution to these workflow problems is to change the fundamental unit of work for production from an issue to an article, something Elsevier recognizes and is currently working toward.

The new article-based e-workflow being developed by Science Direct will streamline interactions between authors, producers, and suppliers. At a management level automation of key functions will yield the following efficiencies: 1) in the e-workflow model, Web sites will be created to automate the electronic submission of articles to an editorial office and to establish an electronic peer review system, and 2) the peer review system will interface with a more automated login and tracking system maintained by the EW.

The new Production Tracking System can then be used by the EW, suppliers, and customers to manage the production and distribution processes more efficiently. Functionally, the EW would also produce two additional intermediary datasets called S100 and S200. These datasets could be sent to the EW for distribution to customers at the time of creation by the supplier and before an S300 dataset was sent to the EW. For example, the physics community, which uses letter journals, would directly benefit by this change in production. Under the e-workflow model, the supplier could immediately upon creation send an S100 dataset that contained a first generation version of the letter or item (i.e., no author corrections) directly to the EW for distribution to a Science Direct Web site. In addition, Science Direct would also be able to distribute content at the article level in the form of an S200 dataset that contained second generation or correct SGML and PDF data. This content would be sent to a Web site before an S300 dataset, representing the entire issue that was sent to the EW by the supplier. It is interesting to note that the EW does not save intermediary datasets once an S300 dataset is created. Pilot projects have been launched to test the e-workflow model.

Finally, it should be noted that as the use of the EW developed and evolved over time, it became apparent -- for operational and customer support reasons -- that some additional support systems would be needed. For example, one of these systems facilitates Elsevier's ability to support customers in auditing the completeness of their collections. Another tracks the history of publications that Elsevier distributes.

In the early 1990s Elsevier Science recognized that production and delivery of electronic content could best be facilitated by conversion of documents to an SGML format. SGML is a tool that enables the rendering of a document to be separated from the content structure of a document. This division is achieved through the use of a document type definition (DTD) and a style sheet. The DTD is a tool by which the structure of a document can be defined through the use of mark-up tags. In addition, the DTD defines a grammar or set of rules that constrain how these tags can be used to mark up a document. A style sheet defines how the content should be rendered, i.e., character sets, fonts, and type styles. Together, these two tools make documents portable across different computer systems and more easily manipulated by database applications. In addition, the separation of content from rendering is also critical to the long-term preservation of electronic scholarly information. That said, the evolution of production and distribution of content by Elsevier Science or the EW has been tightly coupled to 1) the development of a universal DTD for their publications, 2) the successful adoption of a DTD by EW suppliers, and 3) the emergence of the Portable Document Format known as PDF. On average it took two years for all suppliers (at one time greater than two hundred) to integrate a new DTD into production. As inferred from Table I below, by the time one DTD was fully implemented by all suppliers another version of the DTD was being released.

Table I -- DTD Chronology of DevelopmentDTD VersionDateNoteFLA 1.1.0April 1994FLA 2.1.1May 1995FLA 3.0.0November 1995No productionFLA 4.0.0No productionIndex DTD 1.0.0No productionGlossary DTD 1.0.0No productionFLA 4.1.0November 1997Full SGMLFLA 4.2.0February 2000Full SGML PerfectedFLA 4.3.0July 2001FLA 5.0To be announcedXML and MathML

Two types of standards control production exist at the EW, one for the production of content and the other for the distribution of content to customers. On the production side, the standards used are known as PRECAP, CAP, and DTD. For distribution to customers such as Science Direct, the standards are known as Exchange Format for Electronic Components and Text (EFFECT), Electronic Subscriptions Service (EES), and Science Direct On Site (SDOS).

PRECAP, born in 1995, is an acronym for PRE-Computer Aided Production; CAP, born in 1997, is an acronym for Computer Aided Production. The principal differences between the two modes of production are that 1) PRECAP is paper-based and CAP is electronically based, and 2) CAP production produces higher quality content then PRECAP production. In the PRECAP method electronic journals are created from disassembled paper journals that are scanned and processed.

The PRECAP standard was developed for the TULIP project and is still used to produce content today. At first, PRECAP content was distributed for the Elsevier Science EES (1995-1997) using the EFFECT dataset format, a standard that also grew out of the TULIP project. In fact, the standard was first known as the TULIP Technical Specification Version 2.2. Like its predecessor, EFFECT provided a means by which output from PRECAP and CAP processing could be bundled and delivered to customers. The specification defined a map or standard by which component data (e.g., TIFF, ASCII, and PDF) generated by these processes could be accessed by an application or loaded into a database for end-user use. Since its introduction in 1995, the EFFECT standard has gone through several revisions to meet the changing needs of EW customers.[20]

The data components for PRECAP production can consist of page image files, raw ASCII files, SGML files for bibliographic information, and postscript files encapsulated PDF format. Data components for CAP production are only different in that CAP contains no TIFF page images and CAP can produce full SGML instances of an editorial item such as a full-length article. Paper pages are scanned at 300 dots per inch (dpi) to produce TIFF 5.0 (Tag Image File Format) images. The page image has a white background and black characters. Pages are also compressed using the Fax Group 4 standard that can reduce the size of a page image by about 8 percent. The raw text files are generated via Optical Character Recognition (OCR) from the TIFF files. No editing is performed on these files and only characters in the ASCII range 32-126 are present in the content. These raw files are used to create indexes for applications, not for end-user purposes. SGML citation files are created using the full-length article DTD version 3.0 that took nearly three years to develop. DTD version 1.1.0 created in 1994 and version 2.1.1 created in 1995 were considered experimental and were not used to produce content with PRECAP production. Sometime in 1996 or early 1997, PDF files replaced TIFF image files.

The development of the PRECAP standard and its evolution to CAP can be traced in datasets the EW distributed to its customers either as EES or later as SDOS. Between 1995 and 1997, the EW distributed three different versions of datasets to the Elsevier Electronic Subscription Service. These versions are known as EES 1.0, EES 1.1, and EES 1.2. In EES 1.0, datasets contained only TIFF, ASCII, and SGML files. In EES 1.1, PDF files replaced TIFF images. However, in EES 1.2 datasets (around 1997) a new type of PDF file was introduced. This PDF file is called a "true PDF" and is created not from paper but rather from output from electronic type setting. This change marked the birth of the CAP production method. It is important to note that both CAP and PRECAP data can be contained in an EES 1.2 distribution dataset and SDOS datasets. In December of 1998 EES was officially renamed SDOS. From about 1997 to the present, three versions of SDOS datasets (SDOS 2.0, SDOS 2.1, and SDOS 3.0) have been marketed. Like EES, SDOS datasets contain the same component data. Differences in the versions consist of the type of SGML content packaged in the dataset. All versions contain SGML bibliographic data but in version 2.1 tail or article reference data is being delivered in SGML format. It also is important to note that by 1997, using DTD 4.1, EW suppliers produced full-length article SGML. However, this content was not offered to the EW customers from 1997 through 2000. In February of 2000, DTD 4.2 became the production DTD. However, it was only recently, in spring of 2001, that SDOS 3.0 datasets contained full-length article SGML, including artwork files in Web-enabled graphic format. Version DTD 5.0 is now under development but unlike all other DTD versions, 5.0 will be XML-based and MathML-enabled.

This section describes the production processes that take place at Elsevier Science. A later section dealing with prototype development will describe what is necessary to move Elsevier data from the end point of the publisher's production systems (i.e., the CDs containing metadata and other content) into the prototype archive. It rapidly became clear that the bridge between the two worlds -- that of the publisher and that of the archivist -- is a very shaky one. While recognizing that much work has been done with such emerging standards as METS,[21] OAI, OAIS, etc., no archiving standards have yet been universally adopted jointly by major publishers and the academic community. The adoption of such standards is critical to the success of long-term electronic archives, but such standards urgently need further development and testing in a collaborative approach between the academic and publishing communities before they are likely to become an integral part of publishers' workflows.

As the study progressed, it became clear that the lack of accepted standards and protocols to govern such facets of metadata as transmission, data elements, format, etc., would be a major impediment in the future, not only to expanding the prototype but also to realizing the full potential of digital archives.

Major areas of concern regarding the present situation include generic problems associated with introducing "bridgeware" for any given publisher, as well as the unnecessary, if not prohibitive, costs and operational problems associated with developing, maintaining, and operating multiple different sets of bridgeware to accommodate different publishers having different metadata content and formats. This chaotic scenario has the potential to consume inordinate amounts of time and resources to produce, operate, and maintain such multiple instances of bridgeware.

Relating this chaotic environment to our experience in developing the prototype, the team observed that the lack of a commonly agreed upon set of metadata content necessitated the development of data replication and transformation software to convert the data received from the publisher (in Elsevier's case, in EFFECT format) into the format used in the archive itself (in our prototype, OAI and Dublin Core). At Yale, this transformation was made possible by using Extensible Style Language Transformation (XSLT) technology. However, our prototype development represents a far-from-optimal scenario. While it works in the case of Elsevier because the Dublin Core metadata can be generated from the information in the EFFECT datasets, there is no guarantee this would be true for every publisher.

A further danger is that this situation tends to lead to a fragmented approach in which archive information that is additional to what is needed for a publisher's current content operation will be added either as an afterthought at the publisher's end or as a pre-ingestion stand-alone process at the archive's end. However, based on Elsevier's past experiences in TULIP and in the early days of EES, the team observed that the necessary or additional metadata cannot be effectively and satisfactorily produced either as an afterthought post-production process on the publisher's side or as a pre-ingestion conversion activity at the archive's end. Approaching e-archiving in this fashion leads to distribution delays and a more complex production and distribution scenario, with all the accompanying potential to introduce production delays and errors.

The approach to adopt in creating electronic archives should be to recognize at the outset the needs of the archivist, i.e., to collect information to meet these needs as an integral part of the publishing and production process. Archival information will then be subject to the same level of production tracking and quality control as the information gathered in order to provide current-content service.

In their seminal report of 1996, Waters and Garrett[22] lucidly defined the long-term challenges of preserving digital content. Ironically, the questions and issues that swirl about this new and perplexing activity can be relatively simply characterized as a problem of integrity of the artifact. Unlike paper artifacts such as printed scholarly journals which are inherently immutable, digital objects such as electronic journals are not only mutable but can also be modified or transformed without generating any evidence of change. It is the mutable nature of digital information objects that represents one of the principal obstacles to the creation of archives for their long-term storage and preservation. In the CPA/RLG report, Waters and Garrett identified five attributes that define integrity for a digital object: 1) content, 2) fixity, 3) reference, 4) provenance, and 5) context. At a different level, these attributes also represent various impediments or problems to the creation of digital archives. For example, the attribute called context involves, among other issues, the technological environment used to store, search, and render digital objects. Solutions to technological obsolescence such as refreshment, emulation, or migration, are imperfect and can leave digital objects inaccessible -- or even worse, in an irrevocably corrupted state -- if they are not implemented correctly.

Unfortunately, there now exist numerous examples of data that are no longer available to scholars because there is no means to access information stored on obsolete computer hardware and software. In short, technological obsolescence is a vector that can, in a blink of the eye, undermine the integrity of all digital objects in an archive. The YEA was designed to learn how current models, standards, and formats could effectively address the principal problems of integrity and technological obsolescence that threaten the ontology of digital objects. What follows is a narrative that discusses the creation of the YEA, and the findings and lessons learned from our experiment.

Our first step in creating the YEA prototype was to learn about the publisher's technical systems and processes used to create electronic content. Knowledge of the publisher's workflows was important because this knowledge was needed to build an understanding, as defined by the attributes noted above, of digital objects produced by Elsevier Science's EW. At a minimum, the team needed to understand the structure and format of Elsevier Science content and the context or the technical environment needed to read and render the data. In addition, team members needed an understanding of the possible preservation metadata incorporated into Elsevier's data in order to address other issues such as the provenance and fixity of the digital objects that were to be stored in the YEA archive.

Next, team members conducted a review of research projects involved with electronic archives. The findings of this review showed that internationally the Open Archival Information System (OAIS) has been fast-tracked as a model that can be used to describe and create an infrastructure to support the activities of a digital archive. The OAIS reference model specifies and defines 1) an information object taxonomy and 2) a functional model for defining the processes of a digital archival system.[23] The team's other important finding was that the Open Archives Initiative (OAI),[24] an open source model for Web publishing, could be used as a means to access digital objects from the archive.

Archival projects that have adopted the OAIS model include the British Library in London, England; the National Library of the Netherlands, Koninkliijke Bibliotheek (KB) in Amsterdam, the Netherlands; and Harvard University in Cambridge, Massachusetts. In addition, WGBH, a non-profit media organization based in Boston, Massachusetts, plans to build its digital archive based on the OAIS model. The project's technical team decided to make site visits to the European projects and based upon the recommendation of Elsevier Science, to WGBH. Elsevier's technical staff also chose to visit the European sites, both because of Elsevier's relationship with these institutions and because the libraries are exploring the archiving of Elsevier Science content. In addition, both national libraries had already made significant contributions to the study of digital archives. The KB is known for its work on the NEDLIB project[25] and the British Library for its work on preservation description metadata. The site visits proved important and valuable because they provided Elsevier with an opportunity to validate Yale's recommendation that the YEA prototype should be an implementation of the OAIS model. In addition, the team used these visits as a means to explore lessons learned from ongoing digital archive projects. The team wanted to take advantage of project successes and to avoid pitfalls that electronic archiving projects might have encountered. The elements that can contribute to the success or failure of electronic archiving are summarized as follows.

The archival projects at the British Library and the KB represent a continuum of results. The KB has been, by its definition, successful with its project while the British Library has experienced some significant problems with its plans to build a digital archive. Two factors account for the different outcomes. First, the British Library attempted to specify and develop production systems without the benefit of prototype models. In contrast, the KB is developing their archive from a series of research projects that have focused upon building prototypes of future production systems. The other success factor was related to the technical partnership between the library and the vendor chosen to develop the archival systems. In contrast to the British Library, which depended on the vendor to specify application requirements, the KB was able to specify application requirements to the vendor who in turn was able to develop system software. For example, the KB and IBM have successfully begun to modify IBM's Content Manager to integrate into the OAIS model. The KB and IBM have also successfully implemented a data management process and storage management system for digital objects. In addition, the KB and IBM have created beta code for an OAIS ingestion system. Because the KB possessed a deep intellectual understanding of the OAIS model and its requirements, staff were able successfully to exploit IBM's technical know-how. The other valuable lesson came from our site visit to Boston-based WGBH which has successfully partnered with Sun Microsystems to build a digital archive. Through this mutually beneficial partnership, Sun developed a new type of file system for large multimedia objects based upon content owned by WGBH. In return, WGBH was given hardware to support their digital archive. This experience reinforces the value of relevant partnerships that can, among other things, contain development costs.

These site visits also served to validate for YEA that the OAIS and OAI models could serve as foundations for a prototype archive. Consequently, the next objective for the study team was to build small but meaningful components of an OAIS archive. The team speculated, based upon knowledge of Elsevier Science content and Elsevier's EFFECT distribution standard, that rudimentary ingestion, data management, and archival storage processes could be created. Once built, these processes could be engaged to transform a primitive submission information package (SIP) into a primitive archival information package (AIP), which in turn could be searched and rendered via an OAI interface. The prototype-building work began in August 2001 and was successfully completed four months later in December 2001.

In May 2001, sandwiched between the site visit to WGBH and the ground-breaking for the YEA prototype, the team hosted a presentation on digital archiving from the J. P. Morgan Chase I-Solutions group. Chase's I-Solutions potentially offers a means to reduce the total cost of ownership of an archive for a library. That is, Chase's I-Solutions provides archival storage services to commercial businesses that are required to store business transaction data, such as checks and loan agreements, for a minimum period of seven years. Potentially, scholarly archives could outsource the storage management function of their archives to large commercial institutions such as Chase. The Yale team speculated that perhaps, by taking advantage of the scale offered by the Chase archival infrastructure, it might be possible to reduce the total unit cost of storage for a digital object. However, it is difficult to tell at this time whether the cost-saving could be passed on to the customer. While the economics of this concept deserve additional research, other issues make Chase's I-Solutions less useful at this point in time. These issues revolve around trust and major differences between the nature of commercial and scholarly archives. Would the academic community trust a very large commercial institution to archive their content when this community has reservations about relying upon publishers to archive their content? In addition, the I-Solutions archive is currently not designed to capture preservation metadata with stored digital objects. Equally important, the commercial archive is designed to store content for a short period, not the one hundred plus years that academic archives require. Access to the stored objects also poses a problem. Once archived with Chase, an object or its metadata cannot not be altered. Accordingly, at this stage the team believed it was premature to establish a collaborative relationship with Chase.

The Yale/Elsevier technical team perceived that the EFFECT standard and the OAI protocol could be fashioned to build two components of the OAIS model. That is, data in Elsevier's EFFECT format could be transformed into a Submission Information Package (SIP) and the data provider component of the OAI model could be used as an OAIS data management process. Archival storage for the digital objects could be created with a standard UNIX (Solaris) file system. However, the design of the prototype hinged upon the team's ability to convert the EFFECT metadata found in the publisher's SDOS distribution datasets into an XML format. Once converted to XML, the metadata could be transformed into the Dublin Core format that is a requirement for the OAI data provider protocol. Thereafter, the converted metadata could be harvested and exposed by an OAI compliant service provider.

The YEA prototype is split across two different hardware and software platforms. This is idiosyncratic and was done for the sake of expediency. The OAI components are deployed on an IBM Thinkpad model T20 that runs Windows Professional 2000. The Thinkpad has one Intel x86 based, 700 Mhz processor and about 400 Mbytes of memory. The PC's internal disk storage drive has a capacity of 11 Gbytes. The OAIS archival storage component is hosted on a Sun Enterprise 450 workgroup server that run Solaris version 2.8. The system has two (Sun Ultra Sparc-II) 450 Mhz processors and 2 Gbytes of memory. An external Sun A1000 disk array, which provides about 436 Gbytes of storage, is attached to the server.

The OAI data provider software was developed at the University of Illinois at Urbana-Champaign as part of another Mellon foundation project. To run the application the following software is required:

The OAI data store from the University of Illinois-UC conforms to version 1.1 of the OAI protocol. The data service has a simple but powerful design that separates XML metadata files from OAI protocol administrative information about each object. Individual XML files are stored in a Windows file system, and the OAI administrative data are stored in four simple RDBMS (MS Access) tables. In the YEA implementation, the XML files are stored in a MS Windows file system, and the administrative data is stored in an MS Access database. Data in these tables contain the substance of the protocol, without which the data server could not respond to an OAI command or request for information. For example, the metadata table contains all the XML information necessary to describe a metadata object. This information includes the XML namespace, schema, and style sheet information that is needed to transform tagged data in the XML files to Dublin Core. The purpose of the Object table in the database is to maintain a set of pointers to these files. The Repository table provides descriptive information about the repository and the Set table defines collects that are within the repository.

The OAI service provider software is not part of the local environment but was developed at Old Dominion University. The search service called "Arc" is an experimental research service of the Digital Library Research group at Old Dominion University. Arc is used to harvest OAI-compliant repositories, making them accessible through a unified search interface over the network.

Elsevier Science distributes electronic content in a proprietary format that has the fundamental components or structures of a SIP. The standard includes a data object and representation information to provide meaning to the bits that compose the object. As noted above, Elsevier's SDOS datasets are distributed in EFFECT format and contain data objects that are composed of text, image, and PDF files. The encoding standards for each object type are identified and partially defined in the EFFECT dataset. In addition to containing these objects, the EFFECT standard also contains some preservation description information metadata that are needed to maintain the long-term integrity of an archived information object. Finally, the EFFECT standard contains packaging information that 1) uniquely identifies the dataset that contains the information objects so that they can be located through a search process, and 2) provides identification and descriptive information about the delivery vehicle, i.e., the CD-ROM or FTP protocol used to transport the content to a customer or archive. Other representation information in the standard defines the directory structure of the distribution dataset and a logical mapping of how data objects can be assembled to display the digital object through rendering software.

As delivered to Yale, the usefulness of the SDOS data for archival purposes was diminished because these data are not in a normative format such as XML. To transform these data into an XML format, the researchers adopted a piece of software from Endeavor Information Systems (EIS), another company owned by Elsevier Science. EIS was able to process an EFFECT dataset and convert the metadata into XML. Thereafter, the metadata had to be converted to the Dublin Core standard so it would comply with the OAI protocol. Using the expertise of a librarian, the team mapped the EFFECT metadata tags to qualified Dublin Core tags. Once these tags were mapped, an XSLT style sheet was created to dynamically transform EFFECT tags to Dublin Core in response to an OAI request. Approximately fifty items from an SDOS dataset were converted and loaded into the OAI database.

Once data were loaded, the implementation was validated with a tool provided by the Digital Library Research group at Old Dominion University. The tool is called the Open Archives Initiative Repository Explorer.[26] Once the contents are validated, the repository is certified to understand any verb or command in the OAI protocol. The YEA prototype passed this implementation test. After making small modifications to the main application module for the data provider service, the team could experiment with initiating OAI commands to the YEA database. However, the search interface to these data was limited and lacked a feature for retrieving and displaying the digital objects that reside in archival storage.

To provide a unified search interface to YEA, Elsevier Science gave the planning team permission to register the archive with the Arc service provider. Shortly thereafter, Arc's Web harvesting daemon extracted metadata from YEA and, in addition to other repositories, the YEA officially became a repository that could be searched through the Arc interface. Now, not only could YEA metadata be exposed in Dublin Core format, but also links to digital objects in archival storage became active. A reader would not only see metadata about Elsevier Science but could also extract and view the actual content in PDF format through a simple Web browser.

The ability to preserve digital information for long periods of time is dependent upon the effectiveness of the models, standards, formats, and protocols that can be applied to overcome problems associated with technological obsolescence and the maintenance of the integrity of digital objects. The YEA prototype provided some evidence that the OAIS and OAI models can be used to create such an archive. The OAIS model specifies metadata that is needed to preserve the integrity of digital information for long periods of time.

The e-archiving planning team also learned that some of this metadata, e.g., the Preservation Description Information, is already incorporated into distribution datasets disseminated by at least one major publisher, Elsevier Science. However, SIP elements included in Elsevier's EFFECT standard need formally to be developed to include all metadata concerning the fixity, reference, provenance, and context of a digital object. The team recognizes that to obtain these data, production processes used to create Elsevier's content will have to be modified. To be helpful to the archival community, Elsevier needs to assure that its metadata conforms to emerging standards for SIPs.

Recently, Harvard University released a specification for a SIP expressed in XML format.[27] The effectiveness and usefulness of this model can only be determined through robust testing from real applications. Elsevier Science has plans to change its workflows so that content and distribution standards are XML-based. Such a change could provide an opportunity for the planning team, in Phase Two, to work to develop an advanced prototype SIP that conforms to the Harvard specification.

The team's experience with OAI data provider software showed promise for rapid deployment. The economic advantage of using open source software to sustain archives is self-evident. The OAI protocol also showed promise for interoperating with the OAIS model. One shortcoming of a standard OAI implementation is that the Dublin Core standard does not allow all of the rich metadata found in an Information Package to become manifest. Dublin Core was developed as a low-barrier means of searching for data; consequently, it has a limited tag vocabulary. However, the OAI standard does allow for multiple manifestations of an object's metadata. In Phase Two, the Yale-Elsevier team will consider how best to develop an XML schema that can permit the exposure of all metadata found in a formal AIP.

The simplicity of the OAI data store that was implemented suggests that the data management architecture is also scalable and portable. The project team has experiential evidence suggesting that applications that store data objects external to a RDBMS are more flexible, easier to maintain, and more portable across operating systems. These features can potentially make migrations less problematic and more economical when they need to occur.

As alluded to elsewhere in this report, advances in the creation of digital archives are dependent upon systems that can interoperate, and on data and content that are portable. The suite of XML technologies that includes XML, XSL, XPATH, and XSLT provides some hope and high expectations that electronic archives can be successfully architected and implemented. The fact that the project team was able quickly to create an XSLT script with the use of a XML tool demonstrates the promise of already-available technologies. In addition, there is substantial evidence that text data in tagged format such as SGML or XML is very portable across different platforms. The challenge to e-archives is how to make other data formats such as audio and video equally portable across systems. This challenge will be particularly important for specification of standards for Dissemination Information Packages (DIPs).

Finally, the continued success of this impressive beginning between the Yale Library and Elsevier Science to explore digital archives hinges upon a few critical success factors. First is Elsevier Science's continued and unflagging support of Yale Library as a strategic planning partner. The professional relationships and confidence that each institution now has in the other is invaluable. Together, the two institutions are poised to become leaders in the electronic archiving field. Second, Yale and Elsevier need to gain experience with other publishers' formats. In this, Elsevier Science can be helpful, because many of its rendering applications load data from other publishers. Finally, Elsevier Science is interested in exploring the concept of an archival standard that can apply across publishers. Their support and resources could contribute significantly to the development of these systems and standards.

At present, the Yale University Library has several successful components of a Digital Library Infrastructure (DLI) in production use and is actively pursuing additional key initiatives. The Luna Insight image delivery system, for example, elegantly supports faculty and student use of digital images for teaching and study, but it does not yet address the long-term preservation of those images. Over the next three years, we have committed to a substantial expansion of the Luna imaging initiative supported by grant funding from the Getty Foundation, digital collection development in the Beinecke Library, and increased dedication of internal Library resources to the project. Our Finding Aids database provides access to archival finding aids encoded in formats that will endure over time, but, like many of our current stand-alone systems, this resource is not yet tightly integrated into an overall architecture. The participants in the Finding Aids project (including the Beinecke Library, the Divinity Library, the Music Library, and Manuscripts and Archives) continue to add new material to this database and to enhance the public interface.

The Orbis online catalog is perhaps the best example of a system designed, operated, maintained, and migrated with a focus on long-term permanence. The Yale Library staff are currently engaged in the process of migrating from the NOTIS library management system to the Endeavor Voyager system, with production implementation scheduled for July 2002. We are confident that the bulk of our retrospective card catalog conversion effort will also be finished by that date. The meticulous examination of hardware options for the LMS installation has resulted in a high level of expertise among Yale systems staff in the technical requirements for robust, large-scale processing and storage of critical data. The completion of these two fundamental and resource-intensive projects will position the Library well for a substantial investment of energy in innovative digital library initiatives.

In order to enhance reader navigation of digital resources, for instance, we have purchased SFX and MetaLib from Ex Libris. SFX context-sensitive reference linking went into production use at Yale in January 2002 for four information providers: Web of Science, EbscoHost, OCLC FirstSearch, and the OVID databases. Immediately following the Voyager implementation next summer, library staff expect to implement the MetaLib universal gateway in order to deliver unified access, federated searching, and personalized services across a wide range of local and remote resources.

Integration of Library applications with the wider University technical environment is another key principle guiding our work. The Library, for instance, is well advanced in utilizing the campus-wide authentication infrastructure based upon a Central Authentication Service and a universal "NETID." Other areas for collaborative activity include the University Portal project (now in preliminary planning stages), the Alliance for Lifelong Learning,[28] and the Center for Media Initiatives (faculty technology support).[29]

In December 2001, the Collection Management Goal Group (CMGG), one of six strategic planning groups created by the new University Librarian, strongly recommended that the University "establish a Digital Library Infrastructure (DLI) in the Yale University Library worthy of an institution with a 300-year history of acquiring, preserving, and providing access to scholarly research material." There was a powerful conviction in this goal group, and among those consulted by the goal group, that digital preservation is the highest-priority unmet need in the Library's nascent digital infrastructure. The CMGG summarized the first-year objectives as follows:

Senior Library managers are committed to the aggressive pursuit of funding for the creation of this essential digital infrastructure.

NOTE 1: A library willing to negotiate a national site license could function for an entire nation in the manner of a consortial archival agent.

NOTE 2: The access column defines the scope of library operational security provided by each archival agency. For instance, the consortial archival agent ensures permanent online access to all of the publisher's content but only to consortial members, while the publisher-archival agent partnership ensures largely unrestricted permanent online access only to commercially non-viable content.

NOTE 3: The defining concern of the publisher-archival agent partnership, as distinguished from the two other kinds of archival agent, is to identify content that is not commercially viable. Such an agent seeks in this way to minimize conflicts between the commercial mission of the publisher and the preservation/access mission of the archival agent. No boundary between commercially viable and non-viable content has yet been identified. Such boundaries may in time be established by a "rule of thumb" (as has been done with JSTOR or proposed in the case of PubMed Central), or by invoking some set of yet-to-be-specified "trigger events."

NOTE 4: Self-identified archival agents and publisher-archival agent partnerships might provide access to commercially viable content to libraries that once had licenses for that content and/or to libraries in developing countries to which publishers wish, as a matter of good public policy, to provide content for free or at steeply discounted prices.

NOTE 5: Should boundaries between commercially viable and non-viable content be established, consortial and self-identified archival agents may wish to reshape their services with regard to those boundaries, to provide some online access to readers beyond the boundaries of the consortium or physical site. If enough archival agents did this, it is not clear that the publisher-archival agent partnership would have a distinctive function.

NOTE 6: One reason for the maintenance of paper publications is our present inability to identify a boundary between commercially viable and non-viable content. The costs associated with these subscriptions therefore represent, in part, the preservation costs libraries bear. Put differently, the cost burdens of paper publication result, in part, from libraries' unwillingness to put their preservation mission at risk and from publishers' unwillingness to put their commercial mission at risk. This impasse exists because, for online information resources, libraries do not own any information carrier that can be preserved.