ArchivesSpace and Aeon LRW Project

ArchivesSpace and Aeon ETL for Library Data Warehouse (LRW)

Allison Powers, Senior Systems Librarian, Library Technology Services, allison_powers@harvard.edu

Alex Duryee, Service Manager for Archives and Special Collections Systems, alexander_duryee@harvard.edu

Rakesh Kumar, Abotts Inc, rakesh@abottstech.com

Review the ArchivesSpace MySQL database schema and collaborate with Lyrasis to understand data structures, relationships, and access constraints.
Design and develop SQL/PL-SQL queries for full XML data export from the MySQL database, as guided by the data extract specifications provided by ASADWG.
Data extract will include EAD records with MMSID, container records including barcodes, restriction notes,and all other fields as specified by ArchivesSpace and ASADWG requirements. • Design and implement an incremental update process for maintaining current, accurate data in the LRW, including management and reconciliation of deleted records.
Evaluate the feasibility of full data exports for ingest into a Data Lake or Data Lakehouse structure as an intermediary step, to bring data into an SQL-compliant service layer rather than relying on CSV and other flat file models. ASADWG Reporting Requirements ABOTTS will design and deliver reports to support the following ASADWG requirements (requirements currently in process and subject to refinement):

Report on collections with HD storage data.
Report on collections with Alma data, including cross-checking barcodes in Alma and ASpace.
Identify Alma records for collections that do not have corresponding ASpace records.
Report on the Physical Location note field across collections.
Identify outdated restrictions information in collection notes.
Report on Conditions Governing Access notes in use across Alma and ASpace.
Identify DRS links that fail due to http vs. https formatting issues.
Report of name authorities including identification of multiple and duplicate versions.
Create controlled access headings reports.

MySQL ETL Query Set Validated SQL queries for full and incremental data extract from ArchivesSpace
MySQL database Daily Update Pipeline Automated process for incremental LRW updates including deleted record handling
Data Lake Feasibility Report Assessment of full export feasibility to Data Lake / Data Lakehouse with recommended approach
ASADWG Reports All nine (9) reports as specified in Section 3.2, delivered in agreed format Documentation Technical documentation covering ETL design, query logic, update process, and report definitions Assumptions

ArchivesSpace is hosted by the vendor Lyrasis as of January, 2026.

Aeon is hosted by Atlas.