Data Sync Project Outline

Owned by Allison Powers
Last updated: Jan 06, 2022

Phase 1: Reclamation - Complete April, 2021

 Collection analysis 

    • Introduce project with email to primary contact
    • Identify contact person for metadata questions or clean-up 
    • Identify problems in coding and propose remediation steps. Work with local contact to establish a plan of action.

Alma config

    • Create search set = All active holdings on active bibs for library under the OCLC symbol. For example: Berenson holdings = (Library (Holdings) contains phrase (Biblioteca Berenson) AND Tag Suppressed (Holdings) equals "No" AND Tag Suppressed (Title) equals "No"
    • General publishing profiles
      • Publish at holdings level, one record for each Berenson holding in the search set
      • Map holdings to record: 007, 079, 843, 852, 856, 908
      • Normalization rules to code records for OCLC (Under development)
      • Indication rules to create separate files of records (Under development)

WorldCat Collection Manager config

    • OCLC Bibliographic Data Sync collections
      • Create a separate collection for each output file: Manuscripts, Microforms & Electronic reproductions, Vernacular - no trusted OCN, Non-vernacular - no trusted OCN, Trusted OCN.
      • Configure matching type (all match vs. trusted OCN)
      • Preprocessing configuration (if any)

/wiki/spaces/LibraryTechServices/pages/59081106

    • Load OCLC number to holding using cross-reference report as input
    • Load OCLC number to bibliographic record for all but reproduction formats
    • Load 913 fields for Data Sync rejected records

Bibliographic Data Sync Full Export

    • Publish output files from Alma, upload to OCLC
    • Monitor matching and error reports, ongoing corrections and adjustments
    • Upload matching OCLC number to holdings. TBD – how to handle in bibs and holdings for each file type. Remove 079s in holdings not updated by reclamation.
    • Upload reports to web accessible directory
    • Mark sparse records with 908 $ap – or other measures to prevent re-export of records that are never going to match.
    • Scan/delete to remove holdings in WorldCat not added or refreshed by reclamation

Phase 2: On-going Data Sync for new acquisitions - Complete October, 2021

Bibliographic Data Sync Ongoing Exports

Configure on-going export of bibliographic record from Alma for four Bibliographic Data Sync collections. 

    • Publish separate output files for four Bibliographic Data Sync collections
      • Manuscripts – new and updated bibs. Updated bibs are eligible for “own record replace”.
      • Microform and Electronic reproductions – new records for reproduction formats. These are sent separately so the script that loads matching OCLC number can update holding only, not bib.
      • Locally Digitized - new NET holdings for locally digitized materials
      • General – New holdings for all other materials.
    • Collection manager profile(s). Determine whether all output files can use one Data Sync collection relying on filename to distinguish reports, etc. or whether we need separate profiles for each set.

Automate on-going exports, report downloads and processing of report output. Implement WorldCat API for deletions, and a script to remove 014 from holdings.

    • /wiki/spaces/LibraryTechServices/pages/59081106

      • Upload files from dropbox to OCLC ftp server for Data Sync processing
      • Download reports from OCLC ftp server to dropbox directories. Post Bib Processing Reports and Bibliographic Exception Detail Reports in browser accessible directory
      • Upload OCNs from cross reference reports for all four collections to holdings using Alma OCLC (014) Holdings Update
      • Upload OCNs from general and manuscript collections to bibs using import profile
      • Upload OCNs from unresolved record reports for all four collections to bibs using import profile
      • Implement OCLC WorldCat API to delete symbols for holdings/bibs where the OCLC number changes
      • Remove OCN from holdings for records where OCN in bib or holding changes, or is removed

Phase 3: Maintain holdings by implementing Local Holdings Records (LHR) Data Sync - In process

Local Holdings Records Data Sync

Maintains holdings symbols and LHR. Requirements:

      • Valid MARC21 binary
      • Mechanism for exporting from Alma all holdings on a bib when any one of the holdings has changed
      • OCLC number (trusted OCN from holdings)

Configuration - Alma

    • Test Alma pre-configured LHR publishing profile. This job publishes individual records in MARC binary for every holding on a bib where the bib has an 035 with OCLC number, and where the bibliographic management tag is set to ‘Export to WorldCat’.
    • Test Alma general publishing configured to publish at the bibliographic level with embedded data for all linked holdings. When set-up to publish an incremental file, the output should capture all holdings data on a bib on which any holding has changed. Using output from Alma general publishing would require custom scripting to transform content and convert to MARC binary.
    • Configure as necessary depending on publishing process.

Configuration - WorldCat Collection Manager

    • Create translation table – can map at the symbol level, at the library level, at the collection level. Can include mappings for lending and retention based on those levels. This has the potential to help with ILL.
    • Collection configuration – If necessary, configure pre-processing routines to exclude records, remove fields, etc. Would be needed if using Alma pre-configured LHR profile.

/wiki/spaces/LibraryTechServices/pages/59081106

    • Upload files for LHR Data Sync
    • Download error reports and post to web accessible directory
    • Upload LHRN to 990 fields

LHR Data Sync Full Export

    • Full publish LHR from Alma, upload to OCLC
    • Monitor error reports, ongoing corrections and adjustments
    • Post reports to web accessible directory

LHR Data Sync Ongoing Exports

On-going export of holdings that have an OCN.

    • Publish a single file of all holdings on a bib when any one holding has been updated (new, updated, deleted, moved)
    • Input search: holdings PID > 0 and holding suppressed from discovery = No and bibliographic suppressed from discovery = No. Want to send only active, unsuppressed holdings so we don’t need to deal with deleted records.
    • One file – so that all holdings on a bib are always together in the same file.
    • Enable Collection manager profile setting for “Remove WorldCat Holding When Final LHR Deleted” so that symbols are removed when there’s no longer a holding that maps to that symbol.