Asynchronous Processing in Servlets

By default, every web service in INTGeoServer is synchronous: each HTTP request is associated with one and only one thread. If several HTTP requests are being processed at the same time, INTGeoServer keeps as many threads as there are requests in process.

In many use cases, this architecture is sufficient to handle normal loads as each web service is designed to execute quickly. However, if you need to add a long-running web service, asynchronous processing is required to limit the number of concurrent threads and avoid overloading the server. Another use case, typical to AJAX-based applications, is to wait for a status to change on the server. This is called "long polling".

In 2009, the Servlet 3.0 specification introduced the concept of asynchronous servlets. To use asynchronous servlets, you need to use Tomcat 7.0 or greater and enable asynchronous servlets in the web.xml file. Here are the typical steps of a long-polling web service:

• Step 1: the servlet receives the request

• Step 2: the servlet decides to "park" this request in an "asynchronous context"

• Step 3: when the moment is right, the servlet "unparks" this request and "completes" it

The API to park a request is

AsyncContext asyncContext = request.startAsync(request, response);

To decide what to do when "the moment is right", an async listener is attached to that context

AsyncListener asyncListener = ...

asyncContext.addListener(asyncListener);

This AsyncListener interface has 4 methods, one of them is:

public void onComplete(AsyncEvent event) throws IOException;

Deciding when the "moment is right" is up to the application's business logic.

When the "moment is right", the asyncContext.complete() method should be called, triggering all async listeners to execute.

Let's take the example of an application which displays today's lunch menu as soon as it's available. A singleton "MenuManager" has a change listener to notify all interested parties that there is a new menu available.

In step 1, the request handler adds itself to the list of listeners of that menu manager and finishes its execution normally

In step 2: the servlet parks the HTTP request, the client waits for the HTTP answer

In step 3: when the menu manager notifies the servlet that a menu is available, AsyncContext.complete() is called on each AsyncContext

In real-life applications, calling AsyncContext.complete() in the same thread than the thread setting the menu is a bad idea. Java Executors are used instead to make sure that each response has its own thread, without exceeding the maximum number of threads alloted by the application.

Sample Implementation for INTGeoServer

A sample implementation is attached at the bottom of this page. This implementation requires a recent version of INTGeoServer (published after June 12th, 2017) . In this implementation:

1/ the web.xml file has been modified to use the servlet 3.0 specification, and includes the following line:

<async-supported>true</async-supported>

2/ the layer.xml file registers two web services and one context listener

3/ the SetTodaysMenuRequestHandler.java class is the web service setting the menu

4/ the WaitForTodaysMenuRequestHandler.java class is the web service showing the menu, waiting for it if needed

5/ the MenuChangeListener.java executes the pending executions when the menu changes

6/ InitMenuServletContextListener.java is a context listener to initialize the menu manager and the executors pool

Time Out

When you create an asynchronous context, you can specify a time out. If no time out is specified, the default time out of the application server will be used. This time out is typically about 20 to 30 seconds.

To change the time out, call the AsyncContext.setTimeOut method.

If the asynchronous context times out, the following method of AsyncListener is called:

public void onTimeout(AsyncEvent event) throws IOException;

Sample Unit Test

The architecture of INTGeoServer allows testing of request handlers without starting a web server. This applies to asynchronous processing as well. Here is the sample test for this project:

@Test

public void testWaitForMenu() throws Exception {

ObjectNode params1 = JsonNodeFactory.instance.objectNode();

StringWriter stringWriter1 = new StringWriter();

PrintWriter writer1 = new PrintWriter(stringWriter1);

TestServiceContext context1 = new TestServiceContext(params1, writer1, "/waitfortodaysmenu", "/json");

IntGeoServiceContextGlobal.set(context1);

AbstractJSONServiceRequestHandler instance1 = AbstractJSONServiceRequestHandler.Factory.getInstance();

instance1.handleRequest();

ObjectNode params2 = JsonNodeFactory.instance.objectNode();

StringWriter stringWriter2 = new StringWriter();

PrintWriter writer2 = new PrintWriter(stringWriter2);

params2.put("menu", "Fish");

TestServiceContext context2 = new TestServiceContext(params2, writer2, "/settodaysmenu", "/json");

IntGeoServiceContextGlobal.set(context2);

AbstractJSONServiceRequestHandler instance2 = AbstractJSONServiceRequestHandler.Factory.getInstance();

instance2.handleRequest();

Thread.sleep(100); // let executor execute

ObjectMapper mapper = new ObjectMapper();

JsonNode jsonObject = mapper.readTree(stringWriter1.toString());

String menu = JSONUtil.getStringFromJSON(jsonObject, "menu");

assert(menu.equals("Fish"));

}

In this test, we first call the "waitfortodaysmenu" handler. Because the menu is not yet available, the execution is parked until a menu is available. The call to the "settodaysmenu" handler sets the menu, triggering the executors to complete all parked executions.

To simulate a time out, change the Thread.sleep(100) to Thread.sleep(60000)

Passing Parameters

In the example attached at the bottom of this page, the completion of the request doesn't require any parameter. In real-life examples, the handleRequest method might create instances to be read by the onComplete method. The simplest way to pass parameters is to add instance variables to the request handler. As a request handler is created for each received request, this is thread safe.