Notes for outsourced digitization

Selecting an appropriate image file format & specification²

Source material characteristics / type

preferred image file formats (in preference order)

preferred image capture resolutions (in preference order)

Machine printed black and white text documents

The documents include no meaningful color content. The documents do not include color graphics, or color or black and white reproductions of photographs.
The document text is, in all cases, highly legible. The text is black or very dark printed on white or a very light colored paper.
The texts do not include handwritten comments (e.g., pencil annotations).

TIFF format, bitonal (black and white); Group 4 compression. One image file per page-image.
JPEG 2000 Part 1, Core Coding, Lossless Compression format or JPEG 2000 Part 1, Core Coding, Lossy Compression format, 8-bits per RGB channel. One image file per page-image.

600 pixels per inch (ppi), relative to the size of the physical item depicted.
400 ppi
300 ppi

Printed or handwritten documents with color content

Items with meaningful color content, or printed continuous tone images
Discolored documents where the page background has darkened and the contrast between printed or handwritten information and page background is low, e.g.: documents that include light pencil annotations.

Uncompressed, 8-bits per RGB channel TIFF images. One image file per item view.
JPEG 2000 Part 1, Core Coding, Lossless Compression format, 8-bits per RGB channel. One image file per item view.
Color JPEG images saved at a “high” or “highest” quality setting. One image per item view.

400 ppi
300 ppi

Works of art, color photographs

original works of art
printed reproductions
photographic films
lantern slides
glass plate negatives
color and toned monochromatic photographic prints

Uncompressed, 8-bits per RGB channel TIFF images. One image file per item view.
JPEG 2000 Part 1, Core Coding, Lossless Compression format, 8-bits per RGB channel. One image file per item view.
Color JPEG images saved at a “high” or “highest” quality setting. One image per item view.

preference order	Reflective (e.g., prints)	preference order	transmissive (e.g., film)
1.	600 ppi	1.	6000 pixels in long dimension
2.	400 ppi	2.	4000 pixels or greater in long dimension
3.	300 ppi

Monochromatic, black and white photographs, or continuous tone black and white images

Uncompressed, 16-bits per single channel TIFF images. One image file per item view.
JPEG 2000 Part 1, Core Coding, Lossless Compression format, 16-bits per single channel. One image file per item view.
Uncompressed, 8-bits per RGB channel TIFF images. One image file per item view.
JPEG 2000 Part 1, Core Coding, Lossless Compression format, 8-bits per RGB channel. One image file per item view.
Color JPEG images saved at a “high” or “highest” quality setting. One image per item view.

preference order	Reflective (e.g., prints)	preference order	transmissive (e.g., film)
1.	600 ppi	1.	6000 pixels in long dimension
2.	400 ppi	2.	4000 pixels or greater in long dimension
3.	300 ppi

NOTE: ALL COLOR AND CONTINUOUS TONE BLACK & WHITE IMAGES should include embedded ICC display profiles, (e.g., sRGB, eciRGB, AdobeRGB, sGray).¹

Optical Character Recognition (OCR) and keyed text files

For digital objects that include page-images and searchable text, the Harvard Digital Repository Service (DRS) requires deposits include one utf-8 encoded text file for each corresponding page-image file.

For example, a 10 page document deposited to DRS would include 10 image files and identically named (except for the file suffix) text files.

├── 013814337
    ├── 013814337_0001.txt
    ├── 013814337_0002.tif
    ├── 013814337_0002.txt
    ├── 013814337_0003.tif
    ├── 013814337_0003.txt
    ├── 013814337_0004.tif
    ├── 013814337_0004.txt
    ├── 013814337_0005.tif
    ├── 013814337_0005.txt
    ├── 013814337_0006.tif
    ├── 013814337_0006.txt
    ├── 013814337_0007.tif
    ├── 013814337_0007.txt
    ├── 013814337_0008.tif
    ├── 013814337_0008.txt
    ├── 013814337_0009.tif
    ├── 013814337_0009.txt
    ├── 013814337_0010.tif
    ├── 013814337_0010.txt

Naming and organizing files at time of scanning

Prior to scanning documents, one needs to decide how to organize the information so that it can be easily navigated in digital form. Documents have their own organizational structure (individual titles, volumes, issues, chapters, etc.). These meaningful structural components of the scanned documents need to be reflected in the organization of the sequentially numbered scanned page-images arranged within named directories.

Example

At Harvard we might use the HOLLIS ID (The title's bibliographic catalog identifier) as the directory name for a title. It doesn't matter what item ID is used so long as the library's bibliographer or curator has a document key that can be used to unambiguously relate the assigned item ID to a specific title and document description.

File naming restrictions

Filenames must be unique.
Maximum number of characters per file name must be 64 characters or less, and the complete_directory_path + file_name for each file be kept to 255 characters, or less.
Valid characters in file name prefix are letters, digits, underscores ('_'), and hyphens ('-').
File names should not contain spaces.
Use a single '.' character to separate the file name prefix from the file extension. In the case of file compression formats used on archive file formats (e.g. TAR), the double extension format is acceptable. For example: file.tar.gz, file.tar.Z, file.tar.bz2.

Files that share a derivative relationship (e.g., a production master .tif file and its related deliverable .jpg or .jp2 file) should share the same file name in order for Batch Builder to determine that the relationship exists (e.g. clocktower.tif and clocktower.jpg).

Naming schemes: In this example we use three or four components in our directory and file naming.

Components

[Item ID]: At Harvard, we would typically use the Hollis catalog identifier
[Volume ID]: Volume sequence number (multi-volume sets only, 3-digit ID)
[Page sequence number]: Note: four digits
File format (e.g., tif, jpg) extension

Document directory should be named with the item ID (lowercase characters, no spaces)

[002208174] ← this is a directory name: [ITEM_ID]
      |
      | ---- 002208174_0001.jpg
      | ---- 002208174_0002.jpg
      | ---- 002208174_0003.jpg
      | ---- 002208174_0004.jpg
      | ---- 002208174_0005.jpg
      | ...
      | ---- 002208174_0099.jpg
    [007984492]
      |
      | ---- 007984492_0001.jpg
      | ---- 007984492_0002.jpg
      | ---- 007984492_0003.jpg
      | ---- 007984492_0004.jpg
      | ---- 007984492_0005.jpg
      | ...
      | ---- 007984492_0099.jpg

Multi-volume example

[ITEM_ID] ← this is the parent directory for the title.
     |
     | ---- [ITEM_ID]_[VOLUME_ID] ← this is the directory for the volume.
       | ---- 000652831_v001_0001.jpg
       | ---- 000652831_v001_0002.jpg
       | ---- 000652831_v001_0003.jpg
       | ---- 000652831_v001_0004.jpg
       | ---- 000652831_v001_0005.jpg
       | ...
       | ---- 000652831_v001_0099.jpg
       |
     |---[000652831_v002]
     |
       | ---- 000652831_v002_0001.jpg
       | ---- 000652831_v002_0002.jpg
       | ---- 000652831_v002_0003.jpg
       | ---- 000652831_v002_0004.jpg
       | ---- 000652831_v002_0005.jpg
       | ...
       | ---- 000652831_v002_0099.jpg

Providing digital images and associated metadata to external partners (e.g., funders, commercial publishers, other cultural heritage organizations)

Imaging Services will work with Harvard libraries, museums, and archives to share digitized content with external partners.

However, Imaging Services will not:

Produce images or associated metadata files in formats, or to special format-encodings or technical specifications required by external partners when these specification differ standard workflows developed to make digital content that is to be stored by Library's Digital Repository Service.
Responsibilities for transferring data, converting data, and the provision of any supplemental data need to satisfy external partner requirements are to be borne by the owning repository that has contracted with Imaging Services.