Reimagining Discovery

Reimagining Discovery

Vision

Revolutionize how researchers, students, and the global community access and explore Harvard's extensive collections, making all kinds of information easily discoverable and accessible.

Project Goals

  • Enhance user experience 

  • Improve discovery and accessibility of special and archival collections and all types of digital collections including but not limited to image, text, audio, video, born digital, immersive (3d, XR, VR, MR), GIS, etc.

  • Integrate distinct digital collections discovery platforms, including developing a new one

  • Investigate and use AI-powered tools to enhance user experience and metadata

Problem and Value Statements

Problem Statement

Since its founding, Harvard Library has been a guardian of the University’s memory and a gateway to the world's knowledge. We currently host an array of discovery systems that use different design approaches, organizational priorities, and technology standards. Scholars and the public expect to be able to find trustworthy information and discover resources easily regardless of the system that is managing and providing access to it.

Solution Business Value

By enabling rich cross-collection search, this project will offer end users intuitive, contextual discovery of special collections, archives and digital collections, through a mix of conversational interfaces, browsing that emphasizes the visual nature of materials when appropriate, and recommendations for similar or related resources, all informed by ongoing user research.

Alignment with Harvard Library Multi-Year Goals and Objectives

This projects aligns with FY 24 HL Goals:

  • Diversify and expand access to knowledge

  • Maximize the breadth of tangible and digital collections across Harvard and peer institutions, for the benefit of all partners

  • Increase our focus on acquiring, accessing, and creating digital content that is accessible to all, as open as possible, and permits creative uses of collections as data 

  • Invest in open access infrastructure and services that support equitable, sustainable models for scholarly communication and open knowledge

Scope

In Scope

  • Cross collection search for special and archival collections, focusing on the end user experience and making clear the relationships between archival objects/items and larger collections. Deliverables: ArcLight/new HOLLIS for Archival Discovery UI & Collections Explorer.

  • Incorporate AI/ML technologies to offer natural language search, and generative AI features like summarization, while retaining baseline search and browse functions. Deliverable: Collections Explorer.

  • Access to digital content, and act as a replacement HOLLIS for Images and Harvard Digital Collections, extending their use cases to meet project goals: full text searching, born digital, GIS, A/V. Deliverable: to be agreed upon at end of Year 1.

  • Reimagine metadata pipeline using new technologies from AI/ML. Deliverable: Requirements gathering and scoping for a metadata hub. 

Out of Scope

  • Discovery and access to licensed resources (articles, databases) and general collections

Deliverables and Work Products

Key Tasks and Outcomes 

View our Sprint Outcomes and Demos

Definition of Done

Discovery platform, including access to digital assets, is released on production environment and in use by Harvard constituents and the public. 

Stakeholders

Executive Stakeholders

Title

Executive Stakeholders

Title

Martha Whitehead

VP for Harvard Library and University Librarian

Stu Snydman

AUL & Managing Director for Library Technology Services

Salwa Ismail

AUL for Discovery and Access (Jan. 2025)

Tom Hyry

AUL for Archives and Special Collections

The Library Stakeholders are acting as an extended project team, meeting weekly to help inform and prioritize the work.

Library Stakeholders

Title

Library Stakeholders

Title

Amy Deschenes

Head of UX and Digital Accessibility

Kai Fay

Discovery & Access Strategic Projects Manager

Adrien Hilton

Director of Technical Services for Archives and Special Collections

Chelcie Rowell

Associate Head of Digital Collections Discovery

Shalimar Fojas White

Herman & Joan Suit Librarian, Fine Arts Library

Student interns, as needed

Harvard grad and undergraduate students

Technical Project Team

Team Member

Title

Project Role(s)

Team Member

Title

Project Role(s)

Enrique Diaz

Manager of Library Software Engineering

Product Owner (LTS)

JJ Chen

Digital Library Data Engineer

Developer (LTS)

Chris Axon

Senior Digital Library Software Engineer

Developer (LTS)

Maura Meagher

Associate UX Developer

Developer (LTS)

Angela Kilsdonk

Senior IT Project Manager

Project Manager/ Scrum Lead (LTS)

Jenny Rae Bailey

UX Researcher

UX Researcher/Designer (HL)

Past Contributors

Team Member

Title

Project Role(s)

Team Member

Title

Project Role(s)

Katie Amaral

Technical Project Lead

Developer, Architecture (LTS)

Meg McMahon

UX Researcher

UX Researcher/Designer (HL)

Carolyn Caizzi

Senior IT Project Manager

Project Manager/ Scrum Lead (LTS)

Doug Simon

Senior Digital Library Software Engineer

Developer (LTS)

Estimated Schedule

Project is managed by using the Scrum framework and these phases/milestones will be adjusted. Below is a a high level schedule.

 

Phase

Phase Start

Phase End

Completion Milestone

Phase

Phase Start

Phase End

Completion Milestone

1

July 2024

September 2024

Natural language discovery platform with generative AI features for discovering digitized, special and archival collections is built and released to QA for testing.

2

October 2024

December 2024

Platform is tested by end users and improvements are recommended. Research into scaling platform for production is completed. Data pipeline is scoped and work begins. Design process for digitized collections (images) component is completed.

3

January 2025

March 2025

Data pipeline and digitized collections components begin to be built. Consulting with vendor for vector database begins. First iteration of pipeline with Finding Aids and index is created. Decision to soft launch discovery platform is made depending on data pipeline. DECISION: product will not be ready for a soft launch by end of March. 

4

April 2025

June 2025

Finalize data pipeline for Finding Aids and digitized collections components and release to QA (include Alma records for collections that need mediated access if time allows).  Platform is monitored for costs and analytics are gathered and reviewed to plan for full launch September 2025. 

5

July 2025

September 2025

Search relevancy evaluation and tuning; LLM evaluation and implementation of final selection. LLM guardrail implementation. Enhancing system security and observability. Production launch September 15.

6

October 2025

December 2025

Build out pipelines for Alma special collections records. Build out functionality for image records, including front end enhancements, authentication. Continuous improvement of the platform. Metadata Hub discovery completed.

7

January 2026

March 2026

Finalize pipeline for Alma special collections records and deliver record ingest. Implementation of Item Detail Page in Collections Explorer. Enhancements of search result relevancy. Metadata Hub development begins.

8

April 2026

June 2026

Enhancements of search result relevancy. Build out pipeline for image records. Implementation of Image Detail Page, including Viewer and thumbnails. Enhancements to search results and filters. Metadata Hub planning.

9

July 2026

September 2026

Build out pipeline and release of JSTOR Digital Stewardship image records in Collections Explorer. Release of new Collections Explorer front end design based on the new Harvard Library design system.

10-12





Year 3 will focus on building out Metadata Hub, as well as continuously improve the Collections Explorer platform.  Investigation into  and possible rollout of workflows for using AI to improve quality of metadata. 

Assumptions, Constraints, Dependencies, and Risks

Project Assumptions

  • Stakeholders either have or have identified the appropriate subject matter experts to advise on prioritization of work and other project matters

  • Stakeholders will have made available the time required to participate in project activities and to complete tasks as requested

  • Project sponsor and other stakeholders are empowered to make the decision required for the project to be a success

  • Project sponsor will provide written approval to move forward with system development when requested as part of incremental/iterative system demonstrations

Project Constraints

  • Scope - Flexible (all types of digital collections depends on unknowns)

  • Time -  Fixed 3 year project 

  • Cost - Fixed 3 year budget

Project Dependencies

  • ArcLight implementation project

  • Media Presentation Service upgrade

  • LibraryCloud reimagine or defining a new data pipeline

  • DRS Futures project

  • Rapidly changing LLM industry

Project Risks

Description

Plan

Impact

Owner

Description

Plan

Impact

Owner

Rapidly changing Generative AI space

Build system to be flexible, swap out models easily

Cost, trust

Technical Project Team

Library metadata quality is varied and semantic retrieval works with unstructured data

See if metadata fields can help the quality of embeddings; experiment with different embedding models, focusing on full text content and multi-modal models for digital images

Quality of retrieval

Metadata creators and Technical Project Team

Unexpected changes to other library systems like Aeon, JSTOR Forum

Account for and expect changes from external systems in design of data pipeline

Timeline delays

Technical Project Team

Staff capacity to support work of the project

Meeting weekly with stakeholders to ensure there is enough time to plan for bouts of work that include time from broader staff

Overall project success

Library Stakeholders 

Acceptance

Accepted by: Library Stakeholders August 8 2024. In scope deliverables confirmed and defined April 3 2025 by Library Stakeholders and AUL Executive Sponsors. 

Prepared by: Carolyn Caizzi

Effective Date: August 9 2024


We don't have a way to export this macro.