Tutorial: Build an Archive

A number of other button icons are now enabled in the vault editor's toolbar. These buttons let you remove, copy, export, or import the archive. Exporting an archive creates a ZIP file of the archive’s backing store. It offers a way to “share” your archive with another author, who can import the ZIP file you send him (but keep in mind that the ZIP file could be very large for a data archive with thousands of data sets). Note that the copy, export and import operations can be quite time-intensive, particularly for a large data hub. Builder raises a modal “Please Wait...” message dialog to block the UI while these operations are running.

Try It: Launch Builder and create a figure archive

Launch DataNav Builder (follow the launching instructions for your operating system on the download page). The application frame window will appear with the Local Archives locale initially "open", with the vault editor displaying the contents of the archive vault maintained in your workspace directory. If you've never used Builder before, that vault will be empty initially.

Click on the "create new figure archive" toolbar button [Alternatively, you can select the New figure archive command from the Edit menu.] After a very brief delay, a new figure archive thumbnail should appear, with the word "[empty]" centered within it. Notice that the new archive gets the "focus", and that a default title ("Untitled archive"), description ("This figure archive is empty."), and author list ("First A. Lastname") are assigned to it. You'll start adding figures to the archive in the next step.

To add or revise content in a figure archive, you need to "open" the archive in Builder. To do so, click on the archive's thumbnail in the vault editor and a different panel "slides" into view from right to left: the figure archive editor. Since the figure archive is empty, the panel is essentially blank.  Like the vault editor, the figure archive editor includes a vertical toolbar docked to the right-hand side. Click on the round green "+" button in this toolbar (or, alternatively, choose the Edit|Add figure(s)... menu command) to start adding figures to the archive. A file chooser dialog appears. Use it to locate the folder containing the figures you would like to add to the archive, and select the desired figure files (you can use the Shift key to select a contiguous range of items in the file list, or the Ctrl/Command key to make a non-contiguous selection). Click the Select button to confirm your selection. The chooser dialog is replaced by a progress dialog as Builder begins processing the selected files and adding the figures to the archive. Note that you can select both DataNav FypML (.fyp) and Matlab figure (.fig) files in the chooser; Builder will convert any Matlab figure to FypML format before adding it to the archive. When the operation has finished, press the Done button to extinguish the progress dialog.

Make sure you're comfortable moving through the different levels of DataNav Builder's archive-building perspective. Use the chain of buttons in the navigation bar to return to the hub editor and then to the archive vault editor, and then make your way back. Be sure the sample view under construction is on display in the navigation view editor before proceeding to the next step.

Pushing data: The view instance data batch file and Matlab function dn_vibatch().

All of the hub and view construction tasks described thus far are relatively straightforward. Creating a hub and a view and specifying title, author list, and description can all be done most easily in DataNav Builder itself. A view’s template figure is typically prepared in Figure Composer, and this is done only after you’ve really thought about what the view will display, what are the logical categories (the search tags) into which the displayed instance data will be grouped, and so on. Once you’ve created the view, the most difficult task remaining is generating hundreds if not thousands of data instances and associating them with the view.

It would be horribly tedious to do this one instance at a time in DataNav Builder itself; in fact, the application does not offer a means of doing so. Instead, the only way to add instance data to a hub view is to load it from a view instance data batch file. The batch file is a binary, custom-formatted file (.dvb extension) that contains any number of view data instances conforming to an instance configuration that is stored in the file itself. The DataNav Matlab support package includes a utility function that users can call from their own Matlab scripts to create a batch file and add instance data to it: dn_vibatch(). To learn how to use dn_vibatch(), install the M-file in your Matlab command path and type help dn_vibatch at the command prompt.

Try It: Generate a batch file for the sample view

Launch Matlab on your workstation. Change the working directory to the directory in which you installed the DataNav Matlab support package. Among other files, this directory will contain two files specific to this tutorial: template_vu2.fyp and sampleVIBatchScript.m. The latter is the script used to generate the view instance batch file for our sample view. From the Matlab command prompt, run sampleVIBatchScript. Assuming you have configured Matlab's command path and Java class path as instructed in the first step of this tutorial, the script should complete in about 10-20 seconds, generating an instance batch file named forTemplate_vu2.dvb in the same directory. The file will be about 36MB in size.

When you are ready to write your own script to collate data for a real navigation view, take another look at sampleVIBatchScript.m. It could serve as a template for your scripts; at the very least, it will show you how to specify the argument you need to pass to the dn_vibatch() function to create the batch file and load it with instance data.

Once you’ve populated a view instance batch file, you can load its contents into your navigation view from within Builder. Open the relevant view and, in the navigation view editor, click on the blue hub icon with the green arrow [or choose Load view data from the Edit menu]. This will raise a translucent modal dialog that will let you process any number of batch files. Press the “Choose...” button to browse the file system for the batch file. The file chooser will only let you select a batch file if its instance configuration is compatible with the current view’s instance configuration. Two configurations are compatible if they contain the same number of groups and each pair of corresponding groups have matching search tag lists and place holder lists. Once you’ve selected a file, press “Start” to begin loading all of the file’s instance data into the view. A progress bar appears, and the text area will display a message for each data instance parsed from the file, indicating whether or not the instance was successfully loaded. If Builder encounters a data instance in the batch file that has the same set of tag values as an instance already attached to the view, then it will raise a dialog to ask the user what to do: skip the duplicate instance, overwrite the existing instance, skip all duplicates, overwrite all, or stop loading the batch file altogether. When the batch file load has finished, you can repeat the same steps to load instance data from another batch file. Click “Done” to extinguish the dialog.

Try It: Load instance data for the sample view

Click on the "load instance data" icon in the toolbar along the right-hand edge of the navigation view editor. A file browser dialog appears. Navigate to the directory in which you created the view instance batch file forTemplate_vu2.dvb in the previous step of the tutorial; press the "Open" button to select that file. [Note that the dialog's "Open" button will be disabled if the batch file you select has an instance configuration that is incompatible with the configuration for the target navigation view.]

In the blue translucent dialog, you will see the full path to the selected batch file (possibly truncated) next to the "Choose..." button. Press the "Start" button to load the instance data from the batch file. This will take a few seconds. Once the task is complete, press the "Done" button to extinguish the dialog.

Observe that the view is now "operational". A "current instance" is on display in the editor's view canvas, and the control and exp navigation panels reflect the search tag values assigned to the instance data currently on display. To examine other instance data, click on any tag value button to change its value. Alternatively, you can manipulate the navigation controls using only the keyboard. Use the Tab key to move the focus to another control, and the arrow keys to change the current value in that control. Try it!

With this approach, you can easily add more data to a hub view that was created long ago. Suppose you constructed a view for a particular kind of experiment and added some data to it. Much later, you repeat the same experiment on a new subject. To add the new experimental results to the hub, simply process the new data with your existing scripts to construct a new instance batch file, then load the batch file into the hub view within Builder. Now the view will include the results for the new test subject!

Once you have loaded instance data into a hub view, changes to the view’s instance configuration are severely restricted to ensure that the revised configuration remains compatible with the existing instance data. You will not be able to add or remove a configuration group, nor change the list of placeholders and search tags in any existing group. You can, however, change the name or iteration block size of any group, and you can toggle the “iterate” flag for any placeholder within a group.

In the process of constructing a view and populating it with instance data, you may discover you’ve made some mistakes and would like to remove some or all of the instance data. To do so, click on the “Remove...” button in the lower-left corner of a configuration group panel. You can choose to remove only the current instance for the selected configuration group, all instances in that configuration group, all instances in the group for which a search tag is assigned its current value, or the view’s entire instance list.

Zooming in on the displayed view

Whenever you load a new view into the navigation view editor, the rendered view fills the canvas to the extent possible while still maintaining the view's aspect ratio (the graphic is not stretched to fill the canvas, since that could distort it substantially). If you resize the frame window, the rendered view is resized commensurately. This "scale-to-fit" view mode is usually preferred because the entire view is fully visible, and you can always enlarge the frame window to see more detail. On occasion, however, you may want to zoom in on the view; for example, you might want to take a closer look at a small portion of the data traces rendered therein. To do so, simply hold the left (primary) mouse button down near the area you want to magnify, then drag the mouse over that area. Upon depressing the mouse button, you'll notice that the floating navigation panels disappear, the cursor turns into a crosshair, and as you drag the mouse, a gray translucent rectangle follows it around the canvas, anchored on the original mouse-down point. The rectangle's aspect ratio matches that of the rendered view, so don't expect the rectangle's opposite corner to stick to the mouse cursor as you drag. Once you've highlighted the area of interest, release the mouse. The navigation panels should reappear, and the canvas should zoom in on the area you highlighted.

At this point, the canvas is in a "zoom-and-pan" view mode. If you resize the frame window, notice that the graphic within the canvas is not resized; you're merely resizing the window onto that graphic. You can repeat the same mouse gesture to further magnify the view. Alternatively, if you need to pan the graphic in any direction, hold down the right (secondary) mouse button and drag the cursor across the canvas; the graphic should follow the cursor. [Keep in mind that, behind the scenes, the graphic is getting redrawn as you pan; if the view contains large data sets or complex graphics like a heat map, or if your workstation is not powerful enough, the panning motion may be very jerky]. Note that the view will remain in the same scaled and panned state even if you select a different view instance via the navigation panels.

Obviously, you'll want to be able to restore the canvas to the normal "scale-to-fit" mode when you're done examining the fine detail. Note that bottom-most button in the editor's toolbar is enabled after you've zoomed in on the canvas. Click this button to exit "zoom-and-pan" mode and return to "scale-to-fit" mode (the button will be disabled once again).

Try It: Zoom and pan

Follow the instructions in the main text to zoom in on behavioral response traces between 400 and 500 ms in the sample view you created in this tutorial. Then try zooming in even further and panning the view left, right, up, and down. Once you're comfortable with the zoom and pan mouse gestures, click the relevant action button to restore the "scale-to-fit" view mode.

Linking navigation views in the hub navigation map

A practical, useful data archive will typically have a number of related navigation views, each offering a different "window" into the data or a subset of the data stored within the hub's data repository. These views are arranged in a hierarchical fashion in the hub's navigation map, which is displayed graphically in the hub editor. A typical hub will have a single entry-point view that offers a graphical summation of the hub's content. This summary view would be linked to several child views that might present, for example, data supporting key conclusions of the relevant research. These views, in turn, could have their own child views that display the results of specific experiments. The number of views, their content, and their relationships are all decisions that the author must make. DataNav Builder only enforces the top-down structure of the navigation map -- the implication being that the further down the map you go, the more narrowly focussed the view.

Once you have defined more than one view in a single hub, link them together on the hub editor's interactive navigation map canvas. First, put the focus on the parent view. You'll see a downward-pointing blue arrow icon along the bottom edge of that view's thumbnail. Click (don't hold the mouse button down!) that icon and move the mouse. Notice that a gray translucent line follows the cursor, anchored on the bottom of the parent view's thumbnail. Click on the view that you want to become a child of the focussed view. As soon as you release the mouse, the hub navigation map is redrawn to reflect the new link. You can remove the link by clicking on the gray line that represents it, then confirming the removal in the popup window that appears.

When the view currently on display in Builder's navigation view editor is linked to other navigation views in the hub, one additional navigation panel appears below the last configuration group panel. This panel is entitled “Explore Further” and is decorated a little differently than the other navigation panels. It contains a button-like control for each link from a parent view to the current view (a step “up” in the hierarchical navigation map) and for each link from the current view to a child view (a step “down” in the map). If you click on one of these buttons, the current view will fade out, and the selected view will fade in and become the current view displayed and edited in the navigation view editor.

Try It: Link two views in the hub navigation map

Return to the data hub editor, which should display the sample view you've already created. Now repeat the necessary steps of the tutorial (starting from "Create a navigation view") to make a second copy of the sample view. Give the copy a different title, such as "A Copy of My First View", so that you can tell them apart.  Once you've populated the second view with data from the batch file, return once again to the hub editor.