Overview

Use this method to deposit audio files along with additional supporting files containing technical and structural metadata about your audio – such as ADL audio decision lists, AES-57 technical metadata files and AES-60 structural files (playlists) for governing audio playback.

If you don't need to deposit these additional files with your audio, use the procedures described in the sections on Manually Build Audio Objects from Template and Automatically Build Audio Objects from Template to deposit your audio instead. If you want to deposit only audio files and corresponding AES-60 playlists, you may do so using the Audio genre and Text genre in Batch Builder GUI. First deposit the audio and then deposit the playlists as a separate Text object batch. See Manually build audio objects from template and Automatically build audio objects from template for depositing simple audio and Creating Text Object Batches for depositing audio playlists as text objects.

Note that BB2 in advanced mode is COMMAND LINE ONLY. Required and optional metadata such as deposit settings, object and file metadata is supplied using special configuration files.

Default audio properties

Default audio properties for advanced audio are stored in the “/conf/advanced/audio” directory of Batch Builder install directory. These files are never changed under normal circumstances. They govern default metadata assignments for advanced audio. Most default assignments can be overwritten by object-specific, and file-specific properties files.

The following default audio properties files are present in the “/conf/advanced/audio” directory:

• archival_directory.properties
• default.properties
• deliverable_directory.properties
• misc_directory.properties
• playlists_directory.properties
• production_directory.properties

The following settings are set in each file:

archival_directory.properties

• #properties for the archival directory
• usageClass=LOWUSE
• isFirstGenerationInDrs=yes
• role=ARCHIVAL_MASTER

default.properties

• #directories to scan for content. Each directory could have a
• # corresponding _directory.properties file
• dirs=archival,production,deliverable,misc
• cmid=CMID-2.0

deliverable_directory.properties

• #properties for the deliverable directory
• usageClass=HIGHUSE
• isFirstGenerationInDrs=no
• role=DELIVERABLE

misc_directory.properties

• #properties for the misc directory
• usageClass=LOWUSE
• isFirstGenerationInDrs=unspecified

playlists_directory.properties

• #properties for the playlists directory
• usageClass=HIGHUSE
• isFirstGenerationInDrs=unspecified

production_directory.properties

• #properties for the production directory
• usageClass=LOWUSE
• isFirstGenerationInDrs=no
• role=PRODUCTION_MASTER

Command Line Syntax

The existing Batch Builder Windows bat and Unix shell scripts are used to launch Batch Builder in advanced mode. The syntax is:

• On PC: batchbuildercli –advanced audio –d [object directory path]
  
  • Use Windows style directory separator '\'
• On Mac or Linux: ./batchbuildersli.sh -advanced audio –d [object directory path]
  
  • use UNIX style directory separator '/'

Note that by default the object directory name serves as the object Owner Supplied Name (you can override that in audio.properties file).

Example 1 (Windows): batchbuildercli –advanced audio –d "C:\mybatch\004390909"

Example 2 (Mac / Linux): ./batchbuildersli.sh -advanced audio -d /mybatch/004390909

Additional command line optional properties

All command line optional properties for deposit settings, object settings and file settings that are available for "simple CLI" in Batch Builder are also now available in advanced CLI. See more here: https://wikiharvardwiki.harvardatlassian.edunet/confluencewiki/x/X4leCxw2VAg

Procedure

Arrange the content in the directory structure required by Batch Builder

Below is a sample directory structure created in preparation for use with Batch Builder. Underlined directories must be named as shown. Other directories can exist in the directory structure but Batch Builder will ignore them.

/batch/
audio.properties  (required)
 relationships.csv   (required for derivative  relationships)
 mods.xml  (optional;
descriptive metadata for the object)  
/batch/object/
archival/  (required if there are archival master files)
  digiprov.xml  (optional process history for object)
         sub_dir1_Media/ (“_Media” is an optional suffix)
              file1.wav
              file1.xml (AES57 metadata for file1.wav)
              file1.properties (DRS metadata for file1.wav)                
              file2.wav
              file2.xml  (AES57 metadata for file2.wav)
              file2.properties (DRS metadata for file2.wav)
         sub_dir2_Media/ (“_Media” is an optional suffix)
              file1.wav
              file1.xml  (AES 57 metadata for file1.wav)
              file1.properties (DRS metadata for file1.wav)
              file2.wav
              file2.xml  (AES57 metadata for file2.wav)
              file2.properties (DRS metadata forfile2.wav)
    sub_dir1.adl  (ADL for *.wav files in sub_dir1)
    sub_dir2.adl  (ADL for *.wav files in sub_dir2)
      deliverable/  (required if have deliverable files)
         performances/  (required if have deliverable files)
            file1.mp3
            file1.xml  (AES57 metadata for file1.mp3)
            file1.properties  (DRS metadata for file1.mp3)
            file2.mp3
            file2.xml  (AES57 metadata for file2.mp3)
            file2.properties  (DRS metadata for file2.mp3)
         playlists/  (required for playlists)
            playlist1.xml  (AES60 file)
            playlist2.xml  (AES60 file)
      misc/  (optional; contains any # of files in any format)
      original/  (optional)
         audio_object.xml(optional AES57 metadata for original item)
      production/  (required if have production master files)
         sub_dir1_Media/ (“_Media” is an optional suffix)
              file1.wav
              file1.xml  (AES57 metadata for file1.wav)
              file1.properties  (DRS metadata for file1.wav)
              file2.wav
              file2.xml  (AES57 metadata for file2.wav)
              file2.properties  (DRS metadata for file2.wav)
         sub_dir2_Media/ (“_Media” is an optional suffix)
              file1.wav
              file1.xml  (AES57 metadata for file1.wav)
              file1.properties  (DRS metadata for file1.wav)
              file2.wav
              file2.xml  (AES57 metadata for file2.wav)
              file2.properties  (DRS metadata for file2.wav)
         sub_dir1.adl  (ADL for *.wav files in sub_dir1)
         sub_dir2.adl  (ADL for *.wav files in sub_dir2)
      workspace/  (optional; can contain any # of files in any format)

Create the required audio.properties configuration file

This file should be put into the root of the batch directory. The file contains all required batch level and object level settings. The format of it must be:

• US-ASCII text
• Each line in "property=value" format
• Lines can be commented out by starting the line with the '#' symbol.
• Can't use reserved XML characters (see the table below for full list). 
• Table of properties. The table below lists the properties that can be specified in this file. An example audio.properties file is shown in Figure 2.

#properties for the object
alephID=009513791
urnAuthorityPath=FHCL.Loeb
batchName=Project_137_4a
billingCode= FHCL.MUSI.MUS_004
depositAgent=12345678
depositAgentEmail=john_harvard@harvard.edu
failureEmail=example1@fas.harvard.edu,example2@fas.harvard.edu
objectOSN= A299ff
ownerCode=FHCL.MUSI
resourcePattern={n}
successEmail=example1@fas.harvard.edu,example2@fas.harvard.edu
successMethod=EMAIL

Place the audio.properties file at the root of the batch directory, e.g.:

…/my_batch-19a326/
    audio.properties
 

Create relationships.csv file.

Put this file at the root of the batch directory.

This must be a comma-separated file named relationships.csv, supplying relationships between files in the object, and/or between this object and existing DRS objects, and/or between files in this object and existing DRS files or objects.

The file paths inside the file need to be in the form supported by the OS on which you are running Batch Builder (POSIX for MAC/Linux and DOS-style for Windows).

The syntax must be:

[Source_filePath],[relationship_name],[target_type],[target]

where

• • Source_filePath:
    
    • for files in this object: relative path to the source file from the object directory for this object
      
      
  • relationship_name, one of:
    • HAS_SOURCE (indicates derivative relationships between files in this object)
    • HAS_DOCUMENTATION (for pointing from files or the object to DRS-stored documentation files or objects)
    • HAS_METHODOLOGY (for pointing from the object to a DRS-stored methodology object)
    • HAS_RIGHTS (for pointing from files or the object to DRS-stored rights files or objects)

• • target_type, one of:
    • RELATIVE_PATH (used for a relative file path)
    • HUL_DRS_OBJECT_URN (used for the URN of a DRS object)
      For example: urn-3:HUL.DRS.OBJECT:175616
      These are persistent IDs new to DRS2 given to all objects that always have a HUL.DRS.OBJECT authority path
    • OBJECT_OSN (used for the owner supplied name of a DRS object having the same owner code as this object) Note that the object ingest service needs to be updated to support handing relationships using OBJECT_OSN
    • HUL_DRS_OBJECT_ORACLE – DRS ID of the target object
    • HUL_DRS_FILE_ORACLE – DRS ID of the target file
  • Target: relative path to a file, DRS object URN or DRS object owner supplied name
    
    Lines can be commented out by starting the line with the '#' symbol
    
    Sample relationships.csv file. The paths need to be in the form supported by OS you are running Batch Builder on.
    
    #Derivative audio content relationships:
    deliverable\performances\file1.mp3,HAS_SOURCE,RELATIVE_PATH,production\sub_dir1\file1.wav
    deliverable\performances\file2.mp3,HAS_SOURCE, RELATIVE_PATH,production\sub_dir1\file1.wav
    deliverable\performances\file3.mp3,HAS_SOURCE, RELATIVE_PATH,production\sub_dir1\file2.wav
    deliverable\performances\file4.mp3,HAS_SOURCE, RELATIVE_PATH,production\sub_dir1\file2.wav
    
    Place this file at the root of the batch directory, e.g.:
    …/my_batch-19a326/
            audio.properties
        relationships.csv
    
    Note that the following relationships are automatically determined by Batch Builder and should not be listed in the relationships.csv file:
    • HAS_DOCUMENTATION relationships between audio files and audio decision lists deposited at the same time 
    • HAS_PRESENTATION relationships between audio deliverable files and AES60 playlist objects deposited at the same time.

Create MODS file for description metadata (optional)

If manually supplying descriptive metadata, create a MODS file called mods.xml complying with MODS schema versions 3.4 or earlier. See http://www.loc.gov/standards/mods/ for more information about MODS metadata and schemas.

Place this file at the root of the batch directory, e.g.:

…/my_batch-19a326/              
    audio.properties         
    mods.xml         
    relationships.csv

Note that as an alternative to supplying a MODS file, descriptive metadata can be imported from Aleph HOLLIS by Batch Builder by providing an Aleph or Alma ID in the audio.properties file.

Provide technical metadata for files in AES-57 format

Optionally provide technical metadata for files in the batch in the AES57 format.

These files should be named the same as the files they describe, except with an xml file extension. For example in Figure 1 above, the file at

      my_batch-19a326/my_object-A299ff/archival/sub_dir1/file1.xml

contains technical metadata for the file at

      my_batch-19a326/my_object-A299ff/archival/sub_dir1/file1.wav

Batch Builder will read in and use the technical metadata supplied by these files but will supplement it with format metadata supplied by FITS (see http://fits.googlecode.com).

Provide file-level properties files for each file

Optionally provide file-level properties files for files in the batch to override Batch Builder-configured metadata values. These files can contain any file-level batch Builder properties. Each properties file contains properties for a specific audio file. It needs to be placed in the same directory with the audio file the properties of which it is listing. The properties that can be provided in the file-level properties files are shown in the Table below.

Note that because of the way the support for PREMIS rights is implemented in the advanced workflow properties files, you can also set object level embargo and secure storage rights in the audio.properties files. You just need to use the appropriate property names.

These files should be named the same as the files they describe, except with a properties file extension. For example the file at

my_batch-19a326/my_object-A299ff/archival/sub_dir1/file1.properties

would contain metadata for the file at

my_batch-19a326/my_object-A299ff/archival/sub_dir1/file1.wav

Process the batch

See "Batch Builder Execution" section below for batch processing syntax

Batch Builder Execution

Syntax: ./batchbuildercli.sh -advanced audio -d path-to-object-directory
where the path-to-object-directory is a relative or absolute path to the directory containing the content needed to build a single audio object.

For example: 

./batchbuildercli.sh -advanced audio -d /home/prod/content/my_batch-19a326/obj-994533

The program will:

• Create a directory at workspace/moved_content
• Put a copy of the files that were in the misc directory in the workspace/moved_content directory
• Zip up the contents of the misc directory and name it misc_files.zip
• Create the object descriptor for the audio content
  
  • The AES57 file at original/?.xml will be interpreted as technical metadata for the original item to be stored in a <sourceMD> element in the descriptor
  • If any AES60 files were included in the playlist directory:
  • Build separate Text objects for each playlist
    
    • The file name will be used as the object owner-supplied-name
    • Create the object descriptor for each object
    • The descriptor will be named {object-OSN}_descriptor.xml and will be written to the batch directory
    • (Relationships between the playlist object and audio deliverable files will be automatically created by the DRS Loader.)

After executing Batch Builder the following changes should be visible in the batch directory:

/my_batch-19a326/

     batch.xml  (DRS batch loading file)   
     descriptor.xml  (object descriptor for the audio object)   
     my_object-A299ff/      
          misc/         
               misc_files.zip  (files that were in the misc directory)      
          workspace/      
          moved_content/  (contains content in the misc directory)   
     playlist1-A299ff_descriptor.xml  (descriptor for playlist object containing playlist1-A299ff.xml)   
     playlist2-A299ff_descriptor.xml  (descriptor for playlist object containing playlist2-A299ff.xml)

Depositing to the DRS

After executing the script, copy the following portion of the directory structure to your DRS dropbox.

/my_batch-19a326/  (example batch directory)      
     batch.xml      
     descriptor.xml      
     my_object-A299ff/  (example object directory)            
          archival/            
          deliverable/             
          misc/            
          original/            
          production/            
          playlist1-A299ff_descriptor.xml  (example descriptor for a playlist object)            
          playlist2-A299ff_descriptor.xml  (example descriptor for a playlist object)

The DRS loader will not automatically delete any files or directories that are uploaded to a dropbox but that aren't referenced in the object descriptors. These files and directories should be filtered out as part of the transfer process; otherwise they will need to be manually deleted from the dropbox. These include:

• Any other subdirectories in the object directory (e.g. pre-archival, workspace)
• Properties files (audio.properties)
• AES57 files used to indicate metadata for the original item or for audio content files (*.xml)
• MODS file (mods.xml)
• Relationships file (relationships.csv)

Notes:

• Object owner-supplied-names (Object OSNs)
  
  Object OSNs are required in DRS 2 and must be unique within a Owner Code. Please keep that in mind when explicitly assigning objects names, for example in the audio.properties file; or implicitly for example, by the file names of the AES60 playlists.

• URNs
  
  • An object-level URN is automatically given to all objects. These URNs will have a HUL.DRS.OBJECT authority path, e.g.urn-3:HUL.DRS.OBJECT:175844
  • File-level URNs are automatically given to files with the role DELIVERABLE. These URNs will have a local authority path, e.g.urn-3:FHCL.Loeb.Faids:mus00023