Born-digital Processing


FTK Analysis and File Export

(A) Logging into FRED and Mapping Network Drives

Use your own eCommon credentials to log into the FRED. You will then need to map the N: drive:
  1. Go to This PC and click Map Network Drive on the ribbon menu
  2. For Drive choose N: from the drop down menu
  3. For Folder enter the following exactly: \\research.files.med.harvard.edu\countway
  4. Click Finish
  5. You may be prompted to enter your own eCommons login again - if so, please do.

(B) Add Additional Subfolders to E_Proc

Create a folder structure for the FTK analysis and documentation. In N:\E_Proc, in the appropriate creator folder you created during the disk imaging process, create a folder under the Documentation folder entitled: _FTK_Metadata_All_Images

 

         

NOTE: No metadata from the 1st Pass will be retained.

 

(C) FTK: Logging in and Creating New Case

  1. Double-click on FTK 6.0 shortcut on desktop
          **If you receive an error message, reboot the FRED. The database should then be accessible.
  2. A pop-up window labeled Please Authenticate will appear – enter the following information:
    1. User Name: DbAdmin
    2. Password: Obsolete2016
  3. Create a new case within FTK:
    1. Go to Case > New
    2. Fill out the New Case Options pop-up box:
      1. Case Name = creator name (e.g. Folkman, M. Judah; HMS Office of the Dean)
        1. If this is an accrual to a collection, append the accession number to the creator name (e.g. Folkman, M. Judah_Acc. 2017-001)
      2. Reference = collection number or series number
      3. Description
        1. Specify if all images from a collection are being added or if it’s a selection, or if it’s just an accrual
        2. Enter examiner’s name (e.g. All images from H MS c365 added to case. Examiner: Kerr, Meghan)
      4. Description File = skip
      5. Case Folder Directory = defaults to E:\Cases, which is the correct location. FTK will create a subfolder under Cases on the E: Drive (labeled Data) that is labeled the same name you gave in the Case Name field. All metadata, analyses, file exports, and case reports are saved in the case folder on the E: drive.
      6. Database Location = skip
      7. Processing Options > User Templates = defaults to CHoM: 1st Pass.
        1. Click Create
        2. Uncheck dtSearch
        3. Uncheck Registry Reports
        4. Click OK
          1. Now the User Templates will say Custom
    3. Click OK

 

 

(D) Adding Evidence to Case

  1. Window will pop up labeled Manage Evidence
  2. Click Add Acquired Image(s) > OK
  3. Navigate to where the collections’ disk images are saved and select the images, then click Open 

    NOTE: If you end up with a .cue file in addition to the .iso file, choose only the .cue file. FTK will know to pull the rest of the files.

  4. ID/Name: enter the collection or series number
  5. Description: skip
  6. Click OK

A window labeled Data Processing Status: 6.0.0.52 will appear showing status of evidence being added to the case.

The process of adding evidence will be completed when in the pop-up window it says: Process State: Finished. When processing has completed, click Close.


Note: Based on testing, it took roughly 11 minutes for 14 disk images consisting of about 12GB of data to be added to a case. If on the Second Pass, all disk images you are adding say (Processed) next to it, but that it doesn’t say Finished next to Process State, and the original amount of time it took to complete the First Pass has been surpassed, exit out of the Add Evidence window – you should still be able to proceed with the analysis.

WARNING: Once you add evidence to the case, do not change any of the of the folder names in the hierarchy where you are pulling the images from. If you change the any folder names after you've added evidence to the case, the file path will be altered and FTK will not recognize the new file path.

 

(E) First Pass File Examination – Things to Check

  1. Explicit Images Review
    1. Go to Filter in the task bar and from the drop down menu, select Explicit images folder (high score).
    2. If FTK has flagged any files as explicit, review the contents of these files to determine whether or not they need to be acted upon.
    3. After reviewing the high score folder, next select from the same drop down menu Explicit images folder (medium score). Review the files as you did in the previous step.
    4. If after reviewing the flagged files it is deemed that they are not be further acted upon, create a Readme file labeled Readme_Explicit_File_Review in Documentation > _FTK_Metadata_All_Images explaining that after reviewing the contents of the files FTK flagged as explicit, the files are not considered explicit and why.
    5. If explicit files are discovered, contact your supervisor.  
  2. Encrypted Files 
    1. Go to Filter in the task bar and from the drop down menu, select navigate to Encrypted Files 
    2. Scroll down to Encrypted Files and see if the FTK flagged any files as being encrypted.
    3. Determine whether or not these files have been deleted. If so, they will need to be decrypted using PRTK. [Need to add PRTK instructions and that a lot of executable files and programming files come up as encrypted]
    4. If the encrypted files have not been deleted, do not decrypt them until the 2nd Pass.

 

(F) Deleting a Case after Completing 1st Pass

After you have reviewed the evidence to make sure there are no encrypted files or explicit images, go back and delete the case from the database:

  1. From the main FTK screen, Right Click on the case > Select Delete > Click Yes




  2. A Warning screen will then appear – click Yes

If you receive an error message saying you need to manually delete the case:

  • Go to E:drive > Cases > [case name] and delete the case manually
  • Navigate to Recycle Bin on FRED desktop, and permanently delete the case here as well.


(G) Running 2nd Pass

Next, re-run the same case, but this time using the CHoM: 2nd Passprocessing profile:

  1. Create a new case within FTK:
    1. Go to Case > New
    2. Fill out the New Case Options pop-up box:
      1. Case Name = creator name (e.g. Folkman, M. Judah; HMS Office of the Dean)
      2. Reference = collection number or series number
      3. Description = specify if all images from a collection are being added or if it’s a selection, or if it’s just an accrual. Also enter examiner’s name. (e.g. All images from H MS c365 added to case. Examiner: Kerr, Meghan)
      4. Description File = skip
      5. Case Folder Directory = defaults to E:\Cases, which is the correct location. FTK will create a subfolder under Cases on the E: Drive (labeled Data) that is labeled the same name you gave in the Case Name field. All metadata, analyses, file exports, and case reports are saved in the case folder on the E: drive.
      6. Database Location = skip
      7. Processing Options > User Templates
        1. The template defaults to CHoM: 1st Pass. In the drop-down menu, choose CHoM: 2nd Pass
        2. Click Create



          1. Uncheck dtSearch
          2. Uncheck Registry Reports
          3. Uncheck Explicit Images Detection only if NO explicit images were detected in the 1st Pass. Otherwise, keep checked.
          4. Click OK
          5. Now the User Templates will say Custom
            1. Click OK

    NOTE: Explicit Images Detection is included in the 2nd Pass Profile even if the content found has been reviewed and not deemed inappropriate. Including it in the 2nd Pass makes it easier for the processor to bookmark or label these files as restricted.

  2. Adding Evidence to Case
    1. Window will pop up labeled Manage Evidence
    2. Click Add > Acquired Image(s) > OK 
    3. Navigate to where the collections’ disk images are saved and select the images, then click Open

      NOTE: If you end up with a .cue file in addition to the .iso file, choose only the .cue file. FTK will know to pull the rest of the files.

    4. ID/Name: enter the collection or series number
    5. Description: skip 
    6. Click OK

WARNING: Once you add evidence to the case, do not change any of the of the folder names in the hierarchy where you are pulling the images from. If you change the any folder names after you've added evidence to the case, the file path will be altered and FTK will not recognize the new file path.

(H) 2nd Pass Metadata to Save to _FTK_Metadata_All_Images

Only certain automatically generated metadata files will be saved to the _FTK_Metadata_All_Images folder you created earlier in the process.

  1. Go to Computer > Data (E:) > Cases > [case name]
  2. Copy and paste the following files into the _FTK_Metadata_All_Images folder:
    1. The first Excel file in the long list of Excel files at top level. File name begins with FileListing which is followed by the date and a series of numbers and letters: e.g. FileListing_2017-02-01….



    2. Got to Jobs AE



    3. Open the first folder. Copy and paste the following file: JobInformation.log

 


 

 (I) 2nd Pass: Exporting Files

  1. Click Explore tab so you see all of the disk images in the case
  2. You will need to export files from each disk image individually.
  3. Navigate to the first disk image and click + sign next to disk image
  4. Navigate to 2nd highest level in the hierarchy – e.g. highest level is 161.aff, therefore NONAME [FAT12] would be the 2nd level.

    NOTE: If you end up with multiple system file systems (in the case of CDs), navigate to Joliet and only export files from this file system.

  5. Right-click the 2nd highest level in the hierarchy and click Export



  6. An Export window will appear, select the following:



    1. Append extension to filename if bad/absent
    2. Export children
      1.  Preserve Folder Hierarchy
      2. Exclude slack space children files
    3. Create manifest files
    4. Include original path
    5. Items to Include
      1. All Highlighted
    6. Destination base bath: N:\E_Proc\[creator or office name]\UseCopies\[image folder]
    7. Click Ok

  7.  In addition to the extracted files, FTK will also create a text file containing export errors. This text file will be located in the main N: drive electronic records folder for that collection.
    1. For example: N:\E_Proc\FolkmanJudah
    2. The file will be labeled: FTKExportSummary.[case name]_Export



  8. Review the contents of FTKExportSummary.[case name]_Export to determine whether or not it should be retained.
    1. Delete the file if it contains only the following types of common errors:
      1. Object type (File System) is invalid for export
      2. ([unallocated space]) Error getting stream
      3. (ObjectPool) Error getting stream
    2. Save the file to that image’s Documentation folder if it contains errors related to actual files.

  9. After exporting all of the files in the case, right click on the UseCopies folder and select Scan with ESET Endpoint Antivirus

 

(J) IMPORTANT

  • DO NOT delete the 2nd Pass case until you have consulted with the CSA
  • ONLY copy and paste the file EvidenceHistory.log into  _FTK_Metadata_All_Images if you have fully processed the case and are going to delete the 2nd Pass case.

Arranging and describing digital files

Below are instructions-in-progress, please contact the Collections Services Archivist for specific guidance on processing born-digital materials while our policies and procedures are under development. 

Tidying up the UseCopies folder
In the UseCopies folder, you'll find files exported using several different file systems. To streamline access to use copies of files, Center practice is to keep one set of files and delete the rest.  Here are the preferred files system formats (in order of preference):

Drives (internal and external hard drives, USB drives, etc)
  1. FAT32 *
  2. NTFS
  3. exFAT

* One drawback of FAT32 is that it is more limited in terms of maximum size (Single file < 4 GB; single volume < 8 TB). For this reason, you should compare the total size of files within the FAT32 folder and the NTFS folders (just by checking Properties); if the NTSF is greater, then keep both FAT32 and NTSF in the RefCopies folder. 


Optical Media (CDs, DVDs, etc)
  1. Joliet
  2. UDF (Universal Disk Format)
  3. ISO 

Surveying digital files:

After imaging and extracting the contents, the processor should try to open at least one file from every file format present. If the media only contains one file type, for example Microsoft Word documents, sample several. View the files in QuickView, or if they contain video, in FTK. Try to get an idea if the files are correspondence, research data, manuscripts, notes, etc., as well as a date range and any potentially restricted information. Add further guidance following restriction screening recommendations. Add guidance on using FTK tools for high-level analysis, file manifests, folder directories, etc.

Arranging and listing digital files insert guidance from google doc  https://docs.google.com/document/d/1JTBBlKQpXEqq4-h-YujtHdAwTB-C1icc0hHt7h9FHr4/

 How to identify, apply and communicate restrictions:  Amber and Hanna developing recommendations. Review and modify stock access note language (Sedg) - see Yale example in google doc. 

Digital extent: link out to this page/section in manual, maybe explain to to get at it (open use copies, highlight all, right click, properties, etc)  (Sedg)

Processing information: Review and update stock text in EAD template. Add language about whether digital files are presented in the finding aid as their own series/subseries, integrated with physical papers, or both.  (Sedg)

Scope and Content note: Always describe the:

  • Document contents/types (writings, presentations, data, email, etc.)
  • File formats (doc, pdf, jpg, etc.)
  • When using digital-only series, cross reference between related series (ie from a paper correspondence series, say "for additional correspondence in digital format, see the E-mail series")
  • Other aspects you would always note in a scope/content note (major topics/names/organizations etc.)
  • Insert updated sample language.

Guidance/examples on devising series titles (ex: "E-mail" vs. "Digital Correspondence", etc.) (Sedg)

Acquisitions info:  In cases where digital files were transferred from a known device or source, note that here.

Example:   Accession number 2016-127McCormick, Marie C2016 April 06. Digital files transferred from McCormick's personal laptop.

How to label and arrange documentation and use copies on N drive  Insert guidance - clearing digital detritus/system files/unnecessary parent folders, when and how to add series-level folders, do we always keep the e-media labeled folder? etc.  (Sedg)

E-media: now what?

For media you successfully imaged but determined to be blank (or to contain only "unallocated space" or system files), please discard and delete the corresponding images and use-copies:

  1. Note the type and number of pieces of blank media to be discarded in the latest version of your processing plan, and include a similar statement in the finding aid Processing Note.
  2. Note that the media was discarded in the E-media log.
  3. Discard media by using the secure data shredding bin on the 2nd floor in the Access Services area.
For media that failed to image:
  1. Note the type and number of pieces of media you were unable to image in the latest version of your processing plan, and include a similar statement in the finding aid Processing Note.
  2. Also note the media that failed to image in the e-media log.
  3. Do not count the "digital space" on the media towards the final collection digital extent.
  4. Do not list the unimaged disks in the finding aid inventory.


Copyright © 2024 The President and Fellows of Harvard College * Accessibility * Support * Request Access * Terms of Use