Post date: Oct 26, 2014 7:14:13 PM
The following is a rough collection of notes I've gathered so far from working with a mosaic dataset for the SNODAS SWE rasters.
First, why a mosaic dataset? Mosaic datasets are essentially tables of records corresponding to a raster file, dataset, or other mosaic. The record is a link to the actual data, so a mosaic dataset does not copy the source data to a new location, just provides a way to organize a group of independent raster data into a single layer. Additionally, mosaic datasets can be time-enabled, so they can be displayed or queried temporally.
How does the dataset know about time? Short answer: it doesn't. Not initially, at least. Getting the dataset to know about any time attribute of the rasters in the dataset requires proper configuration by the user. First, the user must add a date-type field to the attribute table of the mosaic (accomplished in python using AddField_managment, like any other table). The name of this field is up to the user, but convention suggests "Time". Next, the user must edit that field, and input the dates for each of the raster records in the table. To accomplish this programmatically, one can use an update cursor, but because the field is of date type, one cannot simply write a string to the field. Instead, a datetime object is required. See https://docs.python.org/2/library/datetime.html for more about datetime objects.
In the case of the SWE rasters, the date for each record is in the Name field, which is simply the name of the source file for that record, minus any file extension. That is, the SWE image for March 19, 2010 is SWE_2010031905.img, so the entry in the Name field for that record would be SWE_2010031905. Because of this naming convention, adding the dates for each record was easy, as I could parse the Name field to pull out the date of the raster and build a datetime object to write to the time field. In pseudo-python:
Time = datetime.date(Name[4:8], Name[8:10], Name[10:12]) # builds the date object using the input YYYY, MM, and DD.
(If one is looking to create a time-enabled mosaic dataset with input files that do not have the date in the name, I am not sure what the best way would be to figure out hoe to write the date to the Time field. I suppose, if a list of the file names and dates was available, it could be loaded into a dictionary with the name as the key and date as value, and for every entry in the dataset the dictionary would be queried for that key, returning the date to write to the Time.)
Now, the previous steps get time into the dataset, but do not make the dataset time-aware. To do this, one needs to edit the properties of the dataset. This can be done two ways: (1) right-click the mosaic dataset in a catalog window, and choose properties; or (2) use the Set Mosaic Dataset Properties tool. Both options can be used manually, but the second is required if needing to create a dataset using python. The properties that need tp be changed: Use Time should be enabled, Start Time Field should be the Time field, End time field should be set to Time unless a specific end time field exists, and Time Format should be set to YYYYMMDD.
NOTE: I am unsure of what effect the end time has, except that the service did not work correctly if I left it unset. Additionally, the time format setting may suggest that the Time field does not have to be of date type. This may be true, but when I tried it, I received an error when trying to draw the layers in ArcMap. Enabling time on the layer to use the Time Slider queries the layer using a SQL date query, which does not work with a string format field. I chose the YYYYMMDD time format, but this also may not be necessary: I used it and it is working, so I currently recommend it barring evidence to the contrary.
One last thing that needs to be set if the mosaic is to be used for a time-enabled image service: The Time field needs to be added to the Allowed Transmission Fields, otherwise it is not available in the Image service and the time attributes cannot be queried.
Okay, but how do I create a mosaic dataset in the first place? Mosaic datasets can only be created inside a geodatabase, so make one of those. Then, create the mosaic dataset using the Create Mosaic Dataset tool. Ensure the projection is specified, and select any other options as desired (for instance, the raster data type or band designations can be specified, if necessary for the application). That's all the required steps to get the basic mosaic dataset. This point would be a good time to add a Time field or any other fields.
Now, data can be added to the dataset. The tool to do this is called Add Rasters to Mosaic Dataset. One can specify individual raster files (the raster dataset option), or an entire directory or database for bulk operations (the workspace option). Confusingly, in arcpy, Raster Dataset was the required string option for a workspace; I am unsure about individual files. Under the advanced options, I recommend selecting the Build Pyramids option, the Compute statistics option, and, if wanting to replace existing records, the Overwrite Duplicates option.
Building pyramids for each image was recommended from a blog post I read about the issue. Mosaic datasets can have their own "overviews," but I read these do no work as well. Perhaps they are required if multiple images were to be displayed at once, but this is not the case with the SWE rasters, so I chose to create pyramids for each image, in case they had not previously been created.
The compute statistics option is similar, in that the statistics for an individual raster are calculated if they had not been previously. I hoped this would allow contrast stretches dependent on the data displayed, but this does not seem to be the case (see below); nonetheless, it is probably a good idea anyway.
The overwrite duplicates option allows the mosaic to be easily updated, in case revised data is published and should replace older data (as seems to be the case with the SWE data).
One last thing to note: mosaic symbology is not straightforward. At least, I have not figured it out quite yet. The SWE rasters are typically displayed with a stdev contrast stretch, which requires statistics to be calculated on the layer. However, it seems that any stretch is based on the overall mosaic statistics, not on any given date. I am not sure what to make of this. At the very least, it seems to be necessary to recalculate the mosaic statistics after adding new data; this can be accomplished using the Build Pyramids and Statistics tool, but it has many settings and I do not yet know how best to use it.
Publishing the mosaic dataset: Now that the mosaic dataset is created, it can be published as an image service. To do so, right-click it in catalog and choose Share as Image Service. If seeking a programmatic solution to publishing, I believe one can create a connection to Arc Server, create a service definition file, and publish it, but I have not done this with an image service, so I do not know all the steps involved.
To update a previously-published service, it seems all one needs to do is modify the underlying mosaic dataset, then restart the image service. That is, all updates can be made to the mosaic dataset while the image service is running; it will continue serving the old version. When the updates are finished, stoping and starting the service will refresh it so the changes will be available.