6. Managing Content

Overview

This section describes options for managing your content. You must have the content_editor role to access these features.

Adding files to an object

You can add files to an existing object using Web Admin. For example, you might add a file to replace an existing file of poor quality.

File size limit: "Add file" function currently supports adding files up to 2 GB in size. Files of larger size will be rejected.

MD5 checksums:  Each file added via Web Admin must be accompanied by a 32-digit MD5 checksum created by a third-party application, such as MD5 Summer.  DRS will use the MD5 checksum to verify the integrity of files after they are transferred.  See the DRS User Documentation Center for a list of MD5 applications.

File name rules.  File names must conform to DRS requirements -- no more than 100 characters, can include letters, numbers, period, single underscore, single hyphen.  Do not include spaces in file names. 

Relationships.  Once the file is added, you can relate it to other files or objects already in DRS.  Use options on the Relationships tab.  For example, if the added file is a derivative of another file already in the object, you can create a HAS SOURCE relationship between these files.

To add a file:

1. Using a third-party utility, generate an MD5 checksum for the file. 
2. Display the summary page for the object.
3. Scroll down until you see the Add a File box.
4. Click the Browse button, navigate to the file you want to add, and click Open.
5. Copy the generated MD5 checksum into the Checksum field.
    

6. If the file is first generation (an archival master, for example), enable the First Generation radio button.
   Note: You cannot set two files in the same object to ‘first generation’.

7. Select the DRS Access Flag. Select P for public, R for restricted (Harvard Community only), N for no access (available only to collection managers through Web Admin). Do not select B (Bypass).

8. Select the Usage Class. Select LOWUSE for master files and HIGHUSE for deliverables.

9. Select the Storage Class. 

10. Click the add file button. If the MD5 checksum fails, a checksum error message appears. If the file is added successfully, a confirmation message appears.

11. Adding relationships.  When you add a file in Web Admin, the file Owner Supplied Name (OSN) is set to null and no relationships are created. Only the administrative metadata specified in the “Add a File” dialog is added. The file OSN and any relationships to other files and objects need to be added after the fact. To add these, go to the Admin. Metadata section and Relationships section on the file summary page once the file has been successfully added.  
    
    Here is the procedure for adding a HAS_SOURCE relationship between a deliverable still image you just added and the existing archival master file:

    1. Have on hand the DRS ID of the archival master file.

    2. Open the file summary page for the deliverable file you just added.

    3. Click on Relationships from the menu on the right side.

    4. Click on add relationships.

    5. In the HAS_SOURCE field paste the DRS ID of the archival master file.

    6. Click Add.
       

    7. You will see a HAS_SOURCE relationship appear in the Relationships section of the file, highlighted in green.
        

    8. Click on save changes button to save the new relationship.
    9. You will see a File was successfully updated message.
12. If you added a file to a PDS object, go to the File Structure tab to add the file to object’s structure. See 5.2 Editing file structure (PDS objects) for more information.

Deleting and recovering content

A user with the content_editor role can delete a single object, file or batch from its summary screen or bulk delete a set of content from search results.  Deleted content can be recovered for 60 days.

• Deleting a file deletes the physical file and suppresses file metadata. 
• Deleting an object suppresses the object descriptor and deletes all included files. 
• Deleting a batch deletes metadata about the batch, suppresses the object descriptor(s), and deletes all included files.  Batch metadata cannot be recovered, but objects and files contained within can be recovered for 60 days.

You can delete a batch within 120 hours after deposit.  For batches older than 120 hours, a delete action becomes a deletion request sent to the DRS content manager, who will perform the deletion.

For more information, see these topics:

Delete or recovery from the summary screen
Bulk delete or recovery from search results
Searching for deleted content

Delete or recovery from the summary screen

Deleting a single batch, object or file

1. Display the summary page for the object, file or batch.

2. Click the delete this object | file | batch option in the upper right.
   

   
   

   t

3. When the confirmation screen appears, type an optional reason for the deletion and click the Delete button. For files and objects, a second confirmation screen will appear.  At this time, there is no second confirmation screen when deleting a batch.
4. Click OK to confirm the deletion (objects and files only).
5. Once deleted, the summary screen for the object, file or batch will be marked as deleted. A recover option displays for objects and files (batches cannot be recovered although individual objects and files within a deleted batch can be recovered).
   Note: When an object is deleted, to facilitate redeposit of the object the system will modify the owner supplied name by appending the word “deleted” and the original DRS ID.  Note the modified owner supplied name in the following screenshot of a deleted object’s metadata summary:
   

Recovering a single object or file

Note that deleted batches cannot be recovered, but individual objects and files within a deleted batch can be recovered.

1. On the Search page, retrieve the deleted object or file using an advanced search that includes status=deleted. 
2. Select the object or file to display its summary page.
3. Click the recover this object | file link.  If unrecoverable displays, the 60-day recovery window has expired and the content cannot be recovered.
4. When the confirmation screen appears, type the reason for the recovery and click the Recover button.  A second confirmation screen appears.
   If you really want to recover the object or file, click OK.

Bulk delete or recovery from search results

Note that bulk processing a large set of objects or files can generate a high volume of background transactions -- especially descriptor updates and indexing changes.  These transactions will take time to complete.  There may be extra delay before your changes are visible in search results or individual object and file metadata summaries.

Bulk deletion

You can request bulk deletion of multiple objects and files from search results.  A small-scale bulk delete request -- less than 500 files or one object (containing any number of files) -- will be processed immediately.  A large-scale bulk delete request -- two or more objects; more than 500 files -- will send a request to digital preservation program staff, who will perform the deletion.

1. Perform a search that produces the appropriate results list.
2. Select items for deletion.  Check the box of individual items on the list, or use the Select dropdown to select all items on the page or all items in the result set.
3. Select the Delete objects (or Delete files) option from the Action drop-down menu (to the right and above the Select drop-down).  This opens a delete window. 
4. Enter a Reason for deletion (required).  This is a short note of explanation that will be stored with the deletion event in the associated object’s descriptor.
5. Click Delete (or Request deletion) and then click OK to confirm deletion.
6. Another pop-up window confirms the deletion (or the request for deletion).  Click OK to close the window.
7. When deletion is complete, you will receive an email that lists the amount of content deleted and the reason.  Attached to the message will be text files that list DRS IDs of the deleted content and of the associated deletion events.

Bulk recovery

Content deleted in the previous two months can be recovered.  You can request bulk recovery of multiple deleted objects and files from search results.  A small-scale bulk recovery request -- less than 500 files or one object (containing any number of files) -- will be processed immediately.  A large-scale bulk recovery request -- two or more objects; more than 500 files) will send a request to digital preservation program staff, who will perform the recovery.

Remember that deleted content does not appear by default in Web Admin search results.  To expose deleted content, include “status=deleted” as part of your search query.  It is not possible to use a Quick Search to find deleted content.

1. Perform a search that produces the appropriate results list.  Be sure to include “status=deleted” in the query to expose deleted content.
2. Select items for recovery.  Check the box of individual items on the list, or use the Select dropdown to select all items on the page or all items in the result set.
3. Select the Recover objects (or Recover files) option from the Action drop-down menu (to the right and above the Select drop-down).  This opens a delete window. 
4. Enter a Reason for recovery (required).  This is a short note of explanation that will be stored with the recovery event in the associated object’s descriptor.
5. Click Recover (or Request recovery) and then click OK to confirm.
6. Another pop-up window confirms the recovery (or the request for recovery).  Click OK to close the window.
7. When recovery is complete, you will receive an email that lists the amount of content recovered and the reason.  Attached to the message will be text files that list DRS IDs of the recovered content and of the associated recovery events.

Searching for deleted content

Searching for deleted files and objects:  During the 60-day recovery window, deleted files and objects are still in the DRS file system but do not appear in regular search results.  But you can retrieve deleted files and objects through an advanced search on metadata that includes status=deleted.  Note that deleted content cannot be retrieved by a quick search.

Example 1: Searching by owner supplied name and status=deleted.  Since the system appends data to an object’s owner supplied name at time of deletion, you need to append a wildcard (*) to the owner supplied name for the search to succeed.
 

Example 2: Searching by a specific depositor.  To retrieve all deleted objects deposited by a specific deposit agent, this search combines status=deleted with the metadata value Has Event with Agent ID.  The agent ID is the Wordshack URI for the person who deposited the object.
 

Searching for a deleted batch.  In this release, batch metadata lacks a status value needed for Web Admin to identify a batch as deleted in search results.  This means deleted batches will be included in your batch search results if they match your search criteria. 

Also note that once a batch is deleted, there is a lag in indexing. This means deleted objects and files in the deleted batch can still be retrieved in a search for 5 minutes or more after the deletion.

Downloading content

Users with the content_viewer role can download content from the summary screen of an individual batch, object or file or perform a bulk download of content from search results.  The result of a download request is a zip archive file that you save to a local drive.  Download is a real-time process that begins in your browser as soon as you submit the request.

Are there download limits?  Web Admin does not limit the amount of content you can download, but the amount of local drive space and speed of your network connection will influence how long a download takes and whether it completes successfully.  From a Harvard high-speed wired connection, the transfer rate is an estimated 9 GB per hour.  That means downloading a 50 GB archive across a high speed connection would take approximately 6 hours. 

What unzip tool should I use?  To support downloads larger than 4GB, DRS2 uses the ZIP64 standard.  You will need to use a ZIP client that can handle ZIP64 archive files.  In most cases, the built-in ZIP support available on Windows and Macs does not handle ZIP64 files very well.  (One exception may be downloads of less than 4GB from an individual object or file summary.)

• Windows tools:Most major tools support ZIP64, including PeaZip, 7-ZIP, PK-ZIP, and WinZip.
• Mac ZIP tools:UnArchiver GUI, UnArchiver command line, ExpressZIP GUI, BetterZip GUI, Keka.

Download from the summary screen

You can download a batch, object or file from its summary screen.  The result of a download request is a zip archive file that you save to a local drive.  Download is a real-time process that begins in your browser as soon as you submit the request.

Here is the procedure:

1. Display the batch, object or file summary screen.
2. Click the Download link in upper right.  This opens a download window. 
3. Select the file naming option.  The default “DRS ID” option uses DRS ID as file name and all files will be grouped into a single directory.  The “Original Name” option uses original file names and files will be arranged in the original batch directory structure.
4. Click the Download button to start the download.  While the download is in process, do not close the browser session.  The browser will prompt you to save the archive file when download is complete.

Bulk download

You can request bulk download of multiple objects and files from search results.  The result of a bulk download request is a zip archive file that you save to a local drive.  Bulk download is a real-time process that begins as soon as you submit the request.

Here is the procedure.

1. Perform a search that produces the appropriate results list.
2. Select items for download.  Either check the box of individual items on the list, or use the Select dropdown to select all items on the page or all items in the result set.
3. Select Download Objects (or Files) from the Action drop-down menu (to the right and above the Select drop-down)..  This opens a download window.  
    
4. (Optional) Change the default zip file name if desired.
5. Select the file naming option.  The default “DRS ID” option uses DRS ID as file name and all files will be grouped into a single directory.  The “Original Name” option uses original file names and files will be arranged in the original batch directory structure.  
6. Click the Download button to start the download.  A ZIP file named with the owner supplied name will be saved to a local drive.  While the download is in process, do not close the browser session.  The browser will prompt you to save the archive file when download is complete.