Embedded Files
By Sandra Schloen, May 2017
Here we review some strategies for preparing raster files for use in OCHRE. These might be scanned top-plans from excavations where the primary record was paper-based, or these might be images or maps scanned from printed publications. The goal is to have adequate resolution for viewing the images while keeping file sizes as lean as possible. In any case, always work with a backup of your primary documents and retain high-resolutions copies of your source files for other uses outside of OCHRE or for publication purposes.
Note that we are not discussing primary geospatial images here like satellite images or elevation models (DEM). These have geospatial data embedded in the file format and are thus expected to be large files. Attempts to reduce such files would likely eliminate important data from these files -- you probably don't want to do this!
First of all, to view the properties of an image file, right-click it in the ArcCatalog and select Properties. Alternatively, load it as a Layer in the ArcMap view and right-click the Layer, then select Properties. This gives you details about the file, some of which we will discuss below. Make sure that the file has Spatial Reference information listed in its properties (if it doesn't, the file needs to be georeferenced). Without this information, OCHRE will not be able to display the image in its Map View.
Create Pyramids
A georeferenced tif may have, at a minimum, just a <filename>.tif along with its corresponding "world" file, the <filename>.tfw. If this is the case, it is best to open it in ArcMap, at which point ArcMap will ask if you wish to "Create pyramids." This is always a good idea -- accept this option and several additional sidecar files will be created. This allows OCHRE to display a large tif file more efficiently using the ArcMap tiling techniques built into the toolkit. Alternatively, right-click the image in the ArcCatalog and select Build pyramids. Creating pyramids will add to the bulk of the overall file package for this raster, but it is worth this extra overhead for the sake of significantly better display performance.
Create pyramids? Say Yes!
LZW Compression
OCHRE supports LZW compression and this can significantly reduce the size of an image file. Simply Export the image with this option set appropriately. Alternatively, the Copy Raster tool (described below) gives the option of performing LZW compression on a batch of files.
LZW Compression
Pixel Type and Pixel Depth
Use the Layer, Properties to view the Pixel depth of the image. If it has a 16-bit pixel depth you can halve the size of your image file by reducing this to an 8-bit pixel depth. The impact on the display of the image in OCHRE will be negligible and certainly not worth the price of the larger size. Try this on a test image to verify that you are not losing important image information.
View Pixel Type and Depth
To change the file from 16-bit to 8-bit, search for the "Copy Raster" tool in the ArcToolbox (Geoprocessing menu option, Search for Tools or ArcToolbox).
Use the Copy Raster tool to convert the specified "Input Raster" to 8_BIT_UNSIGNED.
If you need to do this for an entire folder of files, right-click the Copy Raster option in the ArcToolbox (Data Management Tools, Raster, Raster Dataset) to get the Batch option. Use the Environments... button to pop up Environment Settings where you can specify the Workspace for the newly created output rasters.
Batch Copy Raster
From the ArcCatalog, select the folder containing the files to be converted, then click the Next-view button to get the split-view listing of the files (so that they can be selected as a group). Shift-click to select the required batch of file then drag-drop them onto the Input Raster column of the Copy Raster table.
Select the Pixel Type field of the Copy Raster table and set it to 8_BIT_UNSIGNED from the pick-list. Right-click the Pixel type field and select Fill to populate the entire column with this value. Click the "Check values" button -- this will auto-generate the default Output name for each file in the Input list. Click OK to start the Batch process.
NoData Values
There may be areas of a raster that do not represent a data point -- that is, they are "NoData." This may happen, for example, if you georeference a scanned topplan and the process of georeferencing shifts (appropriately) the image to align with the spatial reference. To control the appearance of pixels that represent "NoData" specify a color value for these either when you Save or Copy a Raster. Here the value of 0 was used (or defaulted) resulting in black displaying for NoData-pixels.
A better option is to use a color outside of the normal range; hence we often see the value of 256 used for NoData. This results in no color being assigned, creating an effect as if those areas were transparent.
No Data!
Here, for example, when you georeference an image and finish up with the Rectify option, you can save the file with a NoData value of 256 so that unused pixels do not display inappropriately.
Alternately, if you are adjusting a file that already exhibits this problem, navigate to the file (e.g. the .tif) in ArcCatalog, right click it, choose Properties, then set the NoData values for each band either to 0 (if you are making black represent NoData) or 255 (if you are using white to represent NoData).
Color Bands
Sometimes a tif file is using more "color bands" than is necessary to represent the image. For example, if a black-and-white or grayscale image is using 3 color bands (Red, Green, and Blue) it is using 3 copies of the same information. We can throw away 2 of the "color" bands without losing any information and in the process reduce the file size by 2/3s. That is, a file should not be using redundant color bands unnecessarily.
In some cases there may be a 4th band -- the infrared band, alpha channel, or other data band -- which will not be used by OCHRE. This band can be eliminated without loss of functionality in OCHRE.
View the file in the ArcCatalog -- expand it -- to see its color bands, if any.
View Color Bands
For the 3-band file, for example, drag the Band you want to keep from the ArcCatalog (right-hand) pane to the Layer (left-hand) pane. Re-save (right-click the layer, Data, Export data) the layer as a new raster image. Alternatively, load the image into Photoshop and change the Image Mode to Grayscale.
Note that you can eliminate color by choosing only one of the 3 bands. We noticed that this was useful in the case where an archaeological top plan was drawn on colored graph paper -- sometimes blue graph paper, sometimes green graph paper, sometimes red. For an image that has a backdrop of red graph paper, for example, if you choose to keep just the red color band, it will have the effect of filtering out the red color and minimizing (insofar as it is possible), the effect of the (unwanted) color on the drawing.
Cell Size
The cell size of an image has a big effect on its file size. The smaller the cell size, the bigger (potentially much bigger!) the file size. However, for use within OCHRE, the cell size also determines the resolution of the image to which OCHRE can zoom. If the cell size is not small enough, OCHRE will not be able to zoom in very far. It may be worth experimenting to determine an appropriate cell size for different situations and find a good balance between image resolution (which affects the ability to zoom) and the file size. In ArcMap, if you right-click on a raster layer and select "Zoom to raster resolution" this will determine the extent to which OCHRE will be able to zoom in on the image.
To view the cell size of an image, simply check (right-click on the layer) its Properties. Notice this image has 4 bands, a cell size of 0.002, and is very large (375 MB).
To change the cell size of an image, simply re-save it (right-click on the layer, Data, Export data) with a new cell size. Doubling the cell size to 0.004 still gives us a high degree of resolution for zooming on this image. We are also using the Use Renderer setting and LZW compression. Our uncompressed size has been reduced to 88 MB and we have eliminated the unneeded 4th band. With LZW compression the file is a mere 29 MB!
The Cell size can be changed in batch mode for a group of files. In the ArcToolbox, search for the Raster Resample tool and right-click to get the Batch option. Add the files from the ArcCatalog to the Input window. Specify the Output destination. In the Cell Size field enter the new value. Right-click the field and select Fill to populate the entire column.
Other issues, scientific notation
Sometimes the georeferencing data ends up saved in the associated world file (.tfw) by ArcMap in scientific notation which prevents the image from displaying correctly in OCHRE. If this happens the world file needs to be edited (using any simple text editor) and the scientific notation converted to regular integer/decimal format.
Other issues, images appearing black
Sometimes the image will show up black in ArcMap, in OCHRE and in the Windows file system thumbnail. This may happen for several reasons and there are a few things you can try to have it view normally for OCHRE.
- Check to see if it is in an unexpected compressed format -- you will find the Compression Type on the layer Properties. If so, re-export it (right-click, Data, Export data) in an uncompressed format in order to view it normally. If successful, you can then re-export it using a supported format (e.g. LZW compression).
- Check the Symbology tab of the layer Properties and make sure that Stretch Type isn't set to something inappropriate. Try using None if necessary.
- If the image is a 16-bit grayscale image, try converting to 8-bit pixel depth as described above. In some cases this will fix the black-square problem as it is re-saved as part of this process.
- If you can view the image normally in ArcMap but it shows up black in OCHRE, try re-exporting (right-click, Data, Export data) the image using the "Use Renderer" setting:
Something always to check is whether Windows can display the image thumbnail in the folder View. If it is a black image here, then OCHRE will not be able to display it either.
Google Sites
Report abuse