Open Archival Information System (OAIS) compliant repositories focus on both access and preservation; they serve their designated communities by providing access to content in the short and long terms. When they federate, OAIS repositories improve access in the short term by improving interoperability and introducing external components, such as shared finding aids that accept descriptive information in a uniform package from each repository. (In the HOPE federated model, the Aggregator does not serve as a finding aid itself but delivers descriptive information and sometimes OAIS Dissemination Information Packages (DIPs) to services already used by the designated community.) The following are key characteristics of OAIS federated repositories:
- ''Central Access Site/Node:'' This site independently manages a set of descriptions from many repositories and serves as a union catalog, offering a combined view of the holdings. Users are sent from the central access site/node to local repository sites to retrieve digital objects. Such a system is best supported with a standard set of protocols.
- ''Unique Identifiers:'' In the federation, a repository is responsible for assigning each Archival Information Package (AIP) an identifier which is unique within the larger system. Ideally, such an identifier can store location information to direct consumers to the source repository and/or digital object. The use of web-actionable PIDs not only serves this purpose, but also fosters long-term access.
- ''User Authentication and Access Management:'' If the federated repositories restrict access to some AIPs, there is a need to identify authorized users making requests through the central access site/node. Likewise, if access-based services are extended to the general public, for example charging fees for some content, user authentication should be introduced.
- ''Additional Shared Functionality:'' Repositories may choose to integrate further to share expensive resources: file management for storage, peripheral devices for ingest, or back-up data storage for disaster recovery.
HOPE Federated Architecture
In large part due to the success of Open Access (OA) repositories, which handle research publications and relevant metadata, aggregated digital libraries have become ubiquitous. With a strong emphasis on access, aggregated digital libraries aim to improve the visibility and immediate impact of the research products of their communities. Digital library aggregation solutions have two main functions: 1) harvesting records based on a common schema from local repositories, generally using the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH); and 2) making data available through one or more common finding aids—either through the same service, the model behind DRIVER, or by pushing data to external services. The aggregated digital library model is in some ways very similar to the OAIS federated repositories model; they both offer a common environment for managing data sets coming from various local repositories. They both focus on shared finding aids. Yet aggregated digital libraries put less emphasis on data integrity, fixity, and validity over the long term.
In a sense, HOPE straddles the two. While built on platforms and protocols developed and used by digital libraries, HOPE's content profile of rare cultural heritage materials and its long-term commitment to its local designated community make preservation an imperative. And while preservation remains out of the scope of the present three-year project, it is never out of sight. If not a driving force, the OAIS reference model at least remains a powerful check on HOPE's activities.
The HOPE federated repositories architecture consists of five core components:
- HOPE-Compliant Local Object Repositories (LORs): Local repositories fulfilling a minimum set of requirements that are loosely integrated into a general architecture.
- HOPE PID Service: A web service that supports the requirement for assigning unique identifiers to each AIP within the federated system. PIDs have likewise become a community recognized standard for supporting long-term persistence of web-based resources.
- HOPE Aggregator: A component of OAIS central access. The Aggregator is an access node which accepts descriptive information in a standard package and makes it available to [[Glossary#Discovery Service|discovery services]]. HOPE has split central access into two components, allowing descriptive data sets to be collected in one system and distributed to multiple already existing finding aids.
- Social History Portal and Europeana: A second component of OAIS central access. The Social History Portal is a primary finding aid or central access site, since it is the main portal for the primary designated community. Descriptive data has been optimized for use on the Social History Portal, which will serve as a union catalog for digital and non-digital social history collections. Europeana is currently considered a secondary finding aid. However, if Europeana develops the structures and functionality to present the [[Glossary#HOPE Social History Resource|HOPE Social History Resource]] as a coherent and complex corpus, it may also be considered a primary finding aid.
- HOPE Shared Object Repository (SOR): An optional shared functionality. HOPE's safe storage facility allows content providers to store master files and perform digital object transformations.
As can be seen, HOPE's underlying architecture is modular, loosely coupled, and extensible, allowing for great flexibility in the management of data and content. While the architecture has clear benefits for HOPE's heterogeneous and growing network of local content providers, it sits uncomfortably with the strict data curation doctrine of the OAIS model. OAIS envisions transparent workflows through coherent functional entities: ingest, archival storage, data management, and access, ''inter alia''. It fails to treat in any detail the problem of synchronization across system components handling a single function.
Diagram 2-B shows how the OAIS functional entities are distributed among components in the HOPE system.
Diagram 2-B - OAIS Functional Entities Mapped to HOPE Architecture
Such a complex architecture must rest on well-defined data flows through the whole system to avoid redundancy or discrepancy. In HOPE, this is supported through the heavy use of PIDs to facilitate the exchange of data and through reliance on common data supply protocols. Yet, Local Object Repositories (LORs) remain the linchpin in the system. Flexible and robust local systems are essential and should be extended with tools and applications to support synchronization with the other components. Curation is ultimately the responsibility of local repositories.
Looking more closely at each HOPE component:
HOPE Local Object Repositories (LORs) support the following OAIS functional entities, using a variety of tools and applications:
- ''Pre-Ingest/Ingest:'' assign unique identifiers to submitted objects; validate the integrity of objects and check for viruses; ensure presence of long-term metadata; receive a Submission Information Package (SIP) containing descriptive and administrative metadata as well as digital objects; essentially all Local Object Repositories are responsible for receiving one or more SIPs and packaging them into Archival Information Packages (AIPs) and coordinating updates to archival storage and data management entities; some content providers delegate responsibilities to the HOPE Shared Object Repository (SOR) (see below).
- ''Archival Storage:'' store and manage available Preservation Description Information and in some cases Content Information (in the latter case, error checking, storage management, back up, and disaster recovery may also be performed). Those content providers using the Shared Object Repository manage Content Information with some overlaps with the Shared Object Repository.
- ''Data Management:'' manage Descriptive Information and ensure referential integrity across system components.
- ''Access:'' regulate access to AIPs; deliver Dissemination Information Packages (DIPs) on request.
The HOPE Shared Object Repository (SOR) supports the following OAIS functional entities:
- ''Pre-ingest/Ingest:'' (if needed) assign unique identifiers to submitted objects; validate the integrity of objects and check for viruses; receive SIPs containing Content Information (including the Digital Object itself and Representation Information); perform transformations; package Content Information for AIPs.
- ''Archival Storage:'' store and manage Content Information, performing error checks, back-up, and disaster recovery. AIPs are ultimately stored and managed by Local Object Repositories (see above).
- ''Access:'' deliver DIPs on request.
The HOPE Aggregator supports the following OAIS functional entity:
- ''Access:'' potentially, may deliver DIPs on request.
Related Resources
Consultative Committee for Space Data Systems. ''Reference Model for an Open Archival Information System''. CCSDS 650.0-M-2 Magenta Book. Washington D.C.: NASA, 2012. (https://public.ccsds.org/pubs/650x0m2.pdf)
HOPE: Heritage of the People's Europe. ''HOPE: Mission, Technical Vision and High-Level Design of the Architecture''. August 2012. (http://www.peoplesheritage.eu/pdf/D2_1_Grant250549_HOPE_V2-0.pdf)