Omeka: Metadata Creation & Batch Uploading

These directions describe how to create a metadata spreadsheet and upload a batch of digital objects  (such as PDFs or JPEGs) to http://heritage.lib.pacificu.edu/admin . 

Step 1: Create & upload digital objects (PDFs, JPGs, MP3s, MP4s, etc.)

Create the digital objects

Images (JPGs and/or TIFFs):
We treat items that are primarily visual in nature, e.g. photographs, illustrations, and photos of 3D objects as "Images."

• • Use EpsonScan for flatbed scanning. Use a camera for 3-D objects (Al Weiss has a DSLR camera that we can borrow; or you can simply use a high-quality camera phone.).

  • Straighten, Crop and Correct the image in Photoshop
    1. Open the image in  Photoshop.
    2. To crop & straighten: see this video tutorial
    3. Cropping should generally be just inside the edges of the image, assuming that the original image has straight edges and right angles, like a normal snapshot print. For images where the original object is curved or has a decorative edge, crop just outside the edges.  

  • Save the image as a 400 dpi JPG . Notes: 

    • If scanning: Most items should be scanned at 400 d.p.i. and measure approximately 2,000-3,000 pixels on their longest edge. First scan the item, then open it in Photoshop to check the resolution/size. Very small or very large images may need to be scanned at a higher/lower resolution .

    • If using a camera: You will likely need to export images and then "Save As" a JPG and/or a JPG + TIFF in Photoshop.
      JPGs are what we use for public access in Omeka.
      TIFFs are uncompressed images with higher resolution; we sometimes save them as digital master copies, but we don't upload them to Omeka. See our page on Digital Preservation for more info. 

    • Special Note regarding art scans / images:
      We do not want to make high-resolution versions of copyrighted artwork available to the public, as this could infringe on the artist's rights. So for art:
      - Make a scan at high resolution as described above. This  version will be saved internally on Box for our internal use.
      - Then, "save as" a low-resolution reference image, e.g. 350 pixels on the long edge. This is the version that should be uploaded to Omeka for public use. 

    • For more detailed directions on creating JPEGs, see WCHO Documentation, p. 29

  • Regarding single objects that have 2 or more files associated with it:
    If an item has two or more files associated with it (such as 2 .jpgs for the front and back of a photograph), the items should be saved as separate files and given matching filenames but with different trailing characters, e.g.:
    PUA_MS101_001, PUA_MS101_001b, PUA_MS101_001c
    Multiple files can be attached to the same metadata record. See assigning filenames for more specific information.
    
    

Texts (PDFs):
We treat items that are primarily textual in nature as PDFs, e.g. documents, letters, printed or handwritten texts, news clippings, etc.  as "Texts."

Make PDFs using these tools: 

• • • Canon Copier (downstairs): good for unbound 8 1/2" x 11" documents that can go through a page-feeder.

    • GeniusScan app on a phone or tablet: best for items too fragile or too tightly bound for the scanner.

    • EpsonScan + Adobe Acrobat Pro: for other items when we need high resolution scans. (Note: the scanner tends to make bulky file sizes, so we recommend only using it to make TIFFS/JPGs or to create short, 1-5 page PDFs. )

Save the PDFs in color on the "high quality" setting. If your file sizes seem extremely large (which tends to happen when the original item is printed on textured papers), please ask the Archivist for assistance in tweaking the settings. 

Correct the PDFs (if necessary) using Adobe Acrobat Pro. 

Audio-Visual formats:

We treat items that are primarly video as moving images and audio as sound.

Normally, we outsource the digitization of audio and video files, which are delivered to us on DVD or on a removable hard drive. You will need to save copies of the files in a high-fidelity format (such as WAV or MPEG ), but don't put these on the Omeka server -- they take up too much space. If not already done by our vendor, convert audio and visual files to a compressed format for streaming in Omeka:

• Audio WAV files can be batch-converted to .mp3 format using a small program called LamedropXPd. 

• Video files of various formats can be converted to .mp4 format using Adobe Premiere Pro.

• More detailed information on format conversions and backup locations can be found in the Digital Preservation Plan.
  
  

Save the Digital Files using our Standard File Names

• • Put the files into a temporary folder. You can use the Box server for temp storage if you need to work on multiple computer stations. The location where many digitization projects-in-process are is:
    Box > Library > Archives > Miscellaneous Collections > [Collection Name here]

  • Name the files using these rules: File Names for Digital Objects
    Note: If you need to change the names of a large batch of files at once, the app Bulk Rename Utility can automate the process! This utility only works on files that are on your own computer (not Box). 

Step 2: Create the metadata spreadsheet

• Example Spreadsheet: Prepare the metadata in a spreadsheet using Google Spreadsheets or Excel. You can copy the example and re-use it as a template. 

• Cheat Sheet for Metadata Creation : See this for brief directions for filling out each field.

• Each row in the spreadsheet represents one item in OmekaS. Each column represents a metadata field within that item's record.
  See an example of one item in our digital site which was created from one row in a metadata spreadsheet.

• Sometime, a single field (like "Creator" or "Subject") needs multiple entries. 

  • Don't smash together multiple Creators, Subjects, Locations, etc. When you need to include multiple entries for one field, put all the entries within one cell in your spreadsheet, but separate them with a "pipe" character (and no other spaces):   |

  • For example, if you want to say that a particular article has two creators, it would look like this on your spreadsheet:
    Smith, John|Doe,  Jane

• Sometimes, a single item has multiple digital files associated with it. Only make one row per item, not one row per file. 

  • For example, an oral history recording could have an MP4 of the video and a PDF of the associated transcript;
    or a postcard could have two JPGs, representing the front and back of the card. 

  • Anytime a single item has two or more digital files attached, you should only create one row in the spreadsheet.  

  • See an example of an item created from a single row with multiple files

Step 3: Convert the spreadsheet to CSV format & Upload

Convert to CSV Format

• Go to your metadata spreadsheet and then: Save As > CSV file

• Clean up the CSV file. Tips: 

  • Make sure that your spreadsheet contains records for only one item type (such as "document" or "image"). Only one item type can be uploaded per batch. If you have a mix of PDFs and JPGs that you need to upload, split them into two spreadsheets. 

  • Take out extra columns of metadata that are not needed in the final record

  • Remember that hard returns will not display in Omeka, so do not use them to format your records for legibility. 

• Send the final CSV spreadsheet to the Archivist.
  
  

Upload the Files to Omeka 

• To Batch-Upload all the Files to Omeka:
  Ask the Archivist to do this for you; it requires special log-ins and permissions.
  Some reminders for the Archivist: 

  • Upload files using SFTP (Filezilla). The Systems Librarian and/or UIS should be able to help with access if needed. The server path for uploading files is: heritage.lib.pacificu.edu/files/Sideload/[optional sub-directory]

  • If you put the files into an [optional sub-directory], you will also need to edit: https://heritage.lib.pacificu.edu/admin/ > Modules > File Sideload > Configure > Sideload Directory. The path there should be: /srv/websites/heritage.lib.pacificu.edu/files/Sideload/[optional sub-directory]

  • The CSV spreadsheet must have a column labelled Sideload, and it should contain the filename(s) for each digital object; e.g. PUA_MS101_01.jpg

  • When doing the CSV Import, remember to map the Sideload column as follows:
    Add Mapping (+) > Media Source > Sideload. If more than one file is being attached to a row in the CSV file, also Edit the field to accept multiple values. 

• To upload files one at a time:
  First create records in OmekaS, then attach files manually by going to: Item record > Edit Item > Media Tab > Upload

Save Back-Up Copies 

• Image and PDF files should be backed up into additional locations beyond just being put into Omeka. See  Digital Preservation Plan.