DataNav Server

For advanced users only

Most DataNav users can skip this section. It is geared primarily for developers who work on DataNav, and systems administrators responsible for setting up and maintaining a DataNav portal on a server machine. It includes some details on how DataNav Server works, how to get it up and running within an Apache Tomcat servlet container, and how portal content is persisted on the server's file system.

DataNav Server is the server-side companion to the DataNav Builder and DataNav Viewer clients. It processes every request from a client, queries and/or updates portal contents in accordance with that request, and sends an appropriate response back to the client. It is a relatively simple Java servlet application, consisting of two types of servlets and a singleton "portal server" object.  These entities all run within the context of a web server and Java servlet container; we use the open-source Apache Tomcat container to run DataNav Server in the Lisberger laboratory.

As illustrated in the schematic above, the portal server mediates access to a file-based backing store in which the portal's collection of data and figure archives -- the "archive vault" -- is persisted. The server object handles anonymous requests from DataNav Viewer clients and both anonymous and secure authenticated requests from DataNav Builder clients. Any request to modify portal content or retrieve content from a "private" (not published) archive requires user authentication. The portal entity manages all registered user information (user names and passwords), so it handles user authentication directly rather than relying on the servlet container to do so.

DataNav Server's two servlet classes essentially act as Internet communication conduits between any number of clients and the singleton portal server object. Builder servlets handle all DataNav Builder client requests that access or modify the portal's archive vault or its registered users list, while viewer servlets handle all DataNav Builder or Viewer requests that merely retrieve public portal content. These servlets do little more than validate the request stream content format, or MIME type, then forward the request and response streams to the server object, which must parse the request data and construct a suitable response, accessing and/or modifying the registered users list or the archive vault backing store as necessary. All request-response exchanges passing through the builder servlet use the secure HTTPS protocol, while communications through the viewer servlet are routed over anonymous HTTP. In both cases the MIME type is "application/json; charset=utf-8" -- the request and the response are each encapsulated as a JavaScript Object Notation object, which is a convenient vehicle for data transport for Javascript web clients like the DataNav Viewer. Request-response exchanges must adhere strictly to DataNav's portal communications protocol, which details the format and content of each type of client request and its commensurate server response. Any malformed or invalid request will be rejected by DataNav Server; likewise, an invalid server response will trigger an error message in DataNav Builder or Viewer.

To support simultaneous processing of many client requests, the Java servlet container will typically manage a pool of servlet instances. Each instance can handle multiple client-server request-response exchanges, since each such exchange happens within a separate thread of execution. Naturally, this means that potentially many threads will want access to the underlying backing store at the same time, so data integrity in the face of such concurrency -- particularly when multiple threads are updating portal content -- is a fundamental concern. This, in fact, is why we chose to implement the portal entity as a singleton. All request-response exchanges are handled by this monolithic object, which uses access locks to ensure that only one thread of execution can access the backing store at a time. Of course, this creates a severe processing bottleneck, hampering DataNav Server's responsiveness when many client requests are in the pipeline. The program employs a number of strategies to maximize responsiveness while still preserving data integrity:

• • Modifications of portal content are restricted to the level of entire archives: an archive may mounted, unmounted, or its public/private flag toggled. Finer-grained modifications to an archive's content are allowed only when the archive is checked out by a registered user. The server makes a private copy of the archive accessible only to that user, ensuring that only one user can modify an archive's content at a time.

  • Minimizing the work done behind an access lock. Each request-response exchange thread is broken down into three stages of work: (1) reading and parsing the request stream, (2) accessing the backing store to fulfill the request, and (3) preparing response data and writing it to the response stream. Only step (2) is guarded to ensure single-threaded access. The other two tasks do not interact with the backing store and thus need not block other service threads; in fact, it is crucial that they do not do so because they can often be impacted by significant network delays.

  • In-memory caching of portal content to reduce the frequency of backing store accesses.

The preceding discussion only offers a broad sketch of how DataNav Server works and how it is integrated with the client-side applications of the DataNav suite. Further details on the layout of the portal's backing store, the client-server communications protocol, and installation and configuration issues are found in the subsections of this chapter.