Instructions for QA Testing of Storage Class-Enabled BatchBuilder
8/5/2022 JW DRAFT
On Tuesday Aug 25, 2022 LTS plans to release storage class enhancements to production DRS along with a new version of Batch Builder (BB v2.3.0) that is storage class-enabled. With this release, use of older versions of Batch Builder will not be possible – batches generated in an older version will fail to load because they lack storage class assignments.
In the weeks before the August 23 release, LTS recommends that all depositors test their batch deposit workflows in QA DRS using BB v2.3.0.
Goals of QA testing:
- Verify that your regular deposit processes work as expected using Batch Builder v2.3.0.
- Become familiar with DRS storage classes and how Batch Builder can assign them automatically based on metadata you already provide in the batch.
To request help or report a problem, contact LTS Support.
1. Install Batch Builder 2.3.0
2. Create and Generate Your Test Batches
3. Verify Storage Class Assignment Before Deposit
4. Verify Storage Class Assignment After Deposit
5. Storage Class Assignment Criteria
1. Install Batch Builder 2.3.0
- Download Batch Builder 2.3.0 and follow the installation instructions.
Point Batch Builder at the QA instance of DRS:
On your file system, open the BatchBuilder-2.3.0 folder and change to the \conf subfolder. Locate the bb.properties files:
- Rename the bb.properties file to bb.properties.orig. Click Yes if your system asks you to confirm.
- Rename the bb.properties.qa file to bb.properties. Click Yes if your system asks you to confirm. Your active bb.properties file is now configured for QA DRS. If Batch Builder is open when you do this, you'll need to close and restart it.
- To reverse this action and return to the original bb.properties file (which points to production DRS), just reverse these steps. Rename bb.properties to bb.properties.qa and remove the .orig extension to reactivate the original bb.properties file. Then restart Batch Builder.
2. Create and Generate Your Test Batches
- Create a batch in each content type that you regularly handle. Use your normal procedures and templates. DO NOT manually assign a storage class (in file directory metadata, on the "Optional" tab). Let Batch Builder auto-assign the value. See the Storage Class Assignment Criteria section for auto-assign criteria.
- Try using BB 2.3.0 to create a new project and batch.
- Try using BB 2.3.0 on existing projects created in a prior BB version.
- Important note: A batch created in a prior BB version must be regenerated in BB 2.3.0, otherwise it will fail to load because it lacks storage class assignments.
3. Verify Storage Class Assignment Before Deposit
- After batch generation but before deposit, open an object descriptor.xml file in a text or xml editor and search for examples of a hulDRSAdmin:storageClass element. There should be one of these for every file in the object. Is the assigned storageClass value appropriate for the file? See the Storage Class Assignment Criteria section for auto-assign criteria.
- Examples:
- This storageClass element was assigned to an archival master file that is not deliverable: <hulDrsAdmin:storageClass status="current" creatingAgent="http://idtest.lib.harvard.edu:9020/wordshack/person/43824" createDate="2022-07-28T14:29:39.913Z">AR</hulDrsAdmin:storageClass>
- This storageClass element was assigned to a deliverable file:<hulDrsAdmin:storageClass status="current" creatingAgent="http://idtest.lib.harvard.edu:9020/wordshack/person/102" createDate="2022-07-26T19:14:17.773Z">DE</hulDrsAdmin:storageClass>
- This storageClass element was assigned to a file that is both archival master and deliverable:<hulDrsAdmin:storageClass status="current" creatingAgent="http://idtest.lib.harvard.edu:9020/wordshack/person/102" createDate="2022-08-05T19:17:39.516Z">RE</hulDrsAdmin:storageClass>
4. Verify Storage Class Assignment After Deposit
- Log into QA Web Admin: https://drs2-qa.hul.harvard.edu/drs2_webadmin/search .
- Verify the storage class after deposit.
- Find the file metadata page for the same files you sampled in Step 3.
- Go to the Admin metadata section and look for the Storage class field.
- The value displayed in Web Admin should match the auto-assigned value in the descriptor you checked in Step 3. If values don't match, please report this to LTS.
- Check deposited content that should be deliverable. Try the delivery URN. If delivery links don't work, please report this to LTS Support. Note: It can take up to 15 minutes for content to become deliverable. All newly-deposited files are immediately moved to their preservation storage locations but the deliverable copies are moved into delivery storage by an automated process that runs at every quarter hour.
5. Storage Class Assignment Criteria
Content class | Curatorial/administrative/operational criteria | BatchBuilder auto-assign criteria |
Archival (AR) | Non-sensitive, non-external, non-deliverable/repurposable, non-data, non-basic, non-large file. | File is not deliverable (access flag is not 'P' (public) or 'R' (restricted), has no delivery URN assigned and is smaller than 5 GB in size. |
Basic (BA) | Non-sensitive, non-external, non-deliverable/repurposable, non-data file designated for minimal sustaining management. | No files are eligible for this designation (used for content from the Harvard Google Books Project). |
Data (DA) | Non-sensitive, non-external, deliverable file also designated for read-only access by computational processes. | Not yet implemented (no files are eligible for this designation). |
Deliverable (DE) | Non-sensitive, non-external file designated for patron retrieval. | File is deliverable (access flag = 'P' (public) or 'R' (restricted)) and
|
External (EX) | Non-sensitive file curatorially-designated for external management. | Not yet implemented (no files are eligible for this designation). |
Large (LG) | Non-sensitive, non-external, non-deliverable/repurposable, non-data, non-basic file greater than or equal to 5 GB in size. | File size is ≥ 5 GB. |
Repurposable (RE) | Non-sensitive, non-external, deliverable also designated with an additional archivable role | File is deliverable (has access flag = 'P' (public) or 'R' (restricted) and a delivery URN), but also has file role = 'Archival Master' or 'Production Master'. |
Sensitive (SE) | File designated as containing level 4 high-risk confidential information (HRCI) under the University's security policy (Harvard University, 2019). | Secure storage is not yet implemented. Do not assign Secure Storage Rights metadata to objects in your batch or DRS deposit will fail. |