GIVE Wiki Home‎ > ‎

Participants Manual

GIVE-2: Participants' Manual



This document describes the software infrastructure of the GIVE Challenge. It is targeted at developers of NLG systems who want to participate in the challenge and need to understand how their system will fit into the overall architecture.

The document is structured as follows. In Section #Overview we will give a high-level overview of the GIVE infrastructure. Section #Configuration describes how to build and configure an initial GIVE system that uses two demo NLG systems that come with the distribution. Section #Implementation goes into more detail about the internals of the NLG server infrastructure; it describes how your NLG server is implemented as a Java class, and what utility methods you can use.

During the challenge, we may occasionally release updated versions of the GIVE software. These will come with updated versions of this document, with special emphasis on things that changed. We would also like to draw your attention to the online documentation, in particular the Javadocs that come with the distribution and the Wiki

Architecture overview

The overall architecture of GIVE is split into three components:

  1. the NLG servers, which run on machines provided by the challenge participants (i.e., you);
  2. the 3D client, which runs on the end user's machine; and
  3. the Matchmaker, which runs on a central server provided by the organizers.

A user who discovers GIVE on the Web and decides to give it a try will download the client and run it. This client will connect to the Matchmaker, which randomly selects one of the registered NLG servers and requests a new NLG server instance. The Matchmaker also randomly chooses a game world and sends it to the client and the new server instance. It then tells the client and the server instance how they can connect to each other and closes its connections; from this point on, the user tries to solve the given task following the instructions from the NLG server instance. When the game run is finished -- either because the user solved the task or lost, because the user cancelled the game run, or because the connection timed out, broke down, or the client crashed --, the NLG server instance reconnects to the Matchmaker, uploads a log of what happened in the game run, and terminates. The NLG server is now ready to accept a new connection from a client.

This entire process happens in software that we are providing for the challenge; you should not have to worry about it at all. Your primary job, as the developer of the NLG system, is to implement a Java class derived from give.nlgserver.NlgSystem which gets loaded by our server code. We will describe how to do this in #Configuration. If you ever run into problems that you feel are due to bugs in our part of the code, we urge you to report them in the GIVE participants forum. Although the backbone of the GIVE-2 code has been tested thoroughly in GIVE-1, the newer parts are still unpolished, and we hope to iron out the bugs and make things prettier in time for the challenge, in February.

With the participants' package, you have received a virtual environment (the "development world") that you can use for development purposes. This environment will not be used in the actual challenge in February; we will prepare new worlds (the "evaluation worlds") that no participant will have seen. However, we do promise that the evaluation worlds will not differ in dramatic ways from the development worlds. While there may be new object types, every new object will behave exactly like an existing object. For instance, there might be a piano object; but users will not be able to interact with it, and so it will behave exactly like a couch in the development world. You will get a week's time to adjust your NLG system to the evaluation worlds just before the evaluation period starts. Once the evaluation phase of the challenge starts, you are not supposed to change your NLG server any more. There is nothing we can do to stop you from doing it, but we appeal to your scientific honesty not to gain an unfair advantage over other participants by fitting your system to the evaluation worlds.


Prerequisites and downloads

In order to build your own NLG system, you will need the following software:

  • Java Development Kit, version 5 (aka 1.5) or higher. If you use Linux, Windows, or Solaris, you can download a recent JDK for free from Sun's Java website. MacOS comes with JDK 5 preinstalled.
  • Apache Maven, a Java build tool. You can get it for free here.

You will also need the GIVE-2 tools distribution. The tools distribution is an archive that contains the Matchmaker, the Client, and the development worlds. After you unpack it, the give2-tools-V directory (where V is the version, initially 2.0) should contain the following files:

  • lib/Matchmaker.jar: The Matchmaker
  • lib/Client.jar: The client
  • lib/Worldviewer.jar: A tool for exploring GIVE-2 virtual environments without a network connection
  • lib/Mapviewer.jar: A tool for viewing a GIVE-2 world in a 2D perspective
  • bin: shell scripts for starting the client, matchmaker, and world viewer
  • examples: various example files, including the development world (development-world.giveworld) and a sample matchmaker configuration file
  • lib/lwjgl*: native libraries for the 3D client

Finally, there is a GIVE-2 NLG demo server distribution, which contains two demo servers for GIVE-2. After unpacking, the give2-demo-nlgservers-V directory should contain the following subdirectories:

  • give2-dummy-nlgserver: a minimal NLG server which will simply output plan steps as instructions
  • give2-example-nlgserver: a proof-of-concept NLG server developed by Kristina Striegnitz which will serve as a baseline system in the evaluation

Notice that the NLG server distribution is only meant to get you started on your way to developing your own NLG systems; once you understand how the example servers work, you can delete them. However, you will be using the programs in the Tools Distribution, and when we make changes to the tools in this distribution, you will need to keep your copy updated; otherwise, they may become incompatible with the NLG server code into which your NLG system will plug into. We will announce changes to the Tools Distribution whenever they happen.

Have a look at the development world

As of GIVE 2.1, the GIVE tools distribution contains the Map Viewer, which gives you a top-down view of a GIVE game world. Change into the main directory of the archive you just unpacked, and try this:

java -jar lib/Mapviewer.jar examples/development-world.giveworld

This will bring up a window that displays a 2D view of the development world. If you hover your mouse over an object in the world, you will be shown the name, type, and state of that object. If the object controls some other object, or is controlled by another object -- for instance, many doors are controlled by buttons in that pressing the button opens and closes the door --, the Map Viewer will display an arrow that points from the controlling object to the controlled object.

You can use the Map Viewer either to simply familiarize yourself with a given world. You can also use it as a tool that will support you in developing your own GIVE maps.

Setting up your development environment

In order to familiarize yourself with the GIVE software, you should first build the demo NLG servers and then set up and run your own version of the development environment -- that is, run a demo NLG server, run your own copy of the Matchmaker locally, and run the client in such a way that it points to your local Matchmaker. You can do this as follows.

  1. Start the Matchmaker:
    1. Change to give2-tools.
    2. Copy the Matchmaker configuration file, matchmaker-config.xml, to the current directory from the examples subdirectory.
    3. Start the Matchmaker by typing "./bin/start-matchmaker".
  2. Start the Dummy NLG Server:
    1. Leave the Matchmaker running, and open a new shell.
    2. Change to give2-demo-nlgservers/give2-dummy-nlgserver and run "mvn install assembly:assembly". This will generate a new Jar file, target/give2-dummy-nlgserver-V-jar-with-dependencies.jar. If you have never run Maven before, it will probably download a lot of packages from various places; this is intentional.
    3. Start the Dummy NLG Server by typing "./scripts/start-dummy-nlgserver".
  3. Start the 3D client:
    1. Leave the Matchmaker and the Dummy NLG Server running, and open a new shell.
    2. Change to give2-tools.
    3. Start the client by typing "./bin/start-client localhost". (The "localhost" bit instructs the client to connect not to our central Matchmaker, but to your local one. We ask you to always use a local Matchmaker while you're developing your system; the central Matchmaker is meant for the evaluation phase.)

After the 3D models have been loaded, this should bring up a window in which you can move around with the cursor keys. Follow the tutorial to familiarize yourself with the controls, and then venture out of the first room to receive instructions from the (trivial) NLG system. This is where your NLG system will take over later. Note that you could run the Example NLG Server instead of the dummy server, and this is something you should try; but the example server uses a planner, and will be no fun to use until you've attached a decent planner (see below).

At any point (during, before, or after you're running the client), you can inspect the current status of the Matchmaker and the NLG server over a Web interface. If you open the URL http://localhost:8080/ (replace "localhost" by the name of the machine that runs your Matchmaker), you will see status information about the servers, worlds, and previous game runs that the Matchmaker knows about. Similarly, you can see information about the NLG server on http://localhost:8081. If you inspect the NLG server in this way while a client is connected (ideally from a different machine), you can observe how the NLG server spawns a new instance for this client, and you can watch the messages as the client and the server exchange them.

When you develop your own system, you will generally run your own local copies of the Matchmaker and the client (as above), and point the Matchmaker at your own NLG server. During the evaluation, you will then give us the contact details of your NLG server, and we will point the central Matchmaker at it. Feel free to look at the source code of the demo servers; they are very simple, but should give you an idea about what code you will have to provide.

Configuring the Matchmaker

Now that you have the system running in principle, let's have a look at the configuration files in more detail.

The file matchmaker-config.xml is the configuration file for the Matchmaker. Whenever you start the Matchmaker, it expects to find a file of this name in the same directory. This file consists of the following parts:

  • The main matchmaker element has three attributes that specify ports on which the Matchmaker listens for connections. Unless you have a good reason, you should leave these attributes unchanged. The port attribute specifies the port on which the client will connect to the Matchmaker (by default, 3000). The reporting-port attribute specifies the port on which the server instance will connect to upload the log of the game run (default: 2999). The web-port attribute specifies the port for the Web interface (default: 8080).
  • Within the worlds element, you can specify a list of game worlds. We have preconfigured the matchmaker to read the development world that comes with the distribution.
  • Within the nlgservers element, you can specify a list of NLG servers. Each NLG server has a name that identifies it in the database in which we keep the results of the evaluation runs, a hostname, and a port.
  • Finally, the configuration file may optionally have a database element which specifies the database in which the Matchmaker is supposed to store the evaluation results. The database connection is defined by a username, a password, and a JDBC URL consisting of the database type (we support MySQL and PostgreSQL), the hostname of the database server, and the database name; you can put any database URL here that JDBC understands. See below for details on setting up the database for GIVE.

Notice that if you either omit the database element or the Matchmaker doesn't manage to connect to the given database (this is what happened earlier when you used the configuration file template from the distribution), the Matchmaker will still run; it simply won't store things in a database.

Configuring the NLG server

Now let's look at the configuration file for the NLG server. The NLG server expects a command-line argument that specifies the configuration file. Because this file encodes information about your own system, you need to pay much more attention to the information in it than you did with the Matchmaker. The file contains the following information:

  • The main nlgserver element has three attributes that specify the ports on which your server will listen. The port attribute defines the port on which the NLG server listens for incoming connections from the Matchmaker. Let's say that p is the port you specify here. When clients connect to your server, it will spawn new instances, one for each client. The simultaneous-instances attribute says how many such instances will be open at the same time. Let's say you specify k here. Then the server instances will accept connections from clients on the ports p+1 to p+k. Finally, you can specify the web-port on which your NLG server provides its Web interface.
  • In the nlg-system element, you specify the name of the main class of your implementation of the NLG system. This is always a subclass of NlgSystem.
  • In the matchmaker element, you specify the connection to the Matchmaker. All you have to define here is the reporting port of the Matchmaker; it has to be identical to the one in the Matchmaker configuration. We will use 2999 for the challenge.
  • In the planner element, you specify the planner that should be used to find out how to guide the user to the goal. You specify the actual planner to be used in the style attribute; valid values are ff (for the FF planner), sgplan (for the SGPLAN planner), and lazyff (for the builtin planner). If you specify ff or sgplan, you must download and install the planner executable yourself and specify the complete pathname in the executable attribute, as these planner styles call the planner as an external process.

The planner element is optional -- if you don't specify one, the system uses the builtin lazyff planner by default. As of GIVE 2.2, the "lazyff" is the fastest planner for the GIVE-2 planning problems we are aware of. If you want to use SGPLAN, you can obtain a Linux binary at the IPC-2008 website; we can also give you access to a MacOS version of SGPLAN 5.2.2.

It is up to you how many simultaneous instances you want to allow. There is a tradeoff: If you allow many simultaneous instances, you will collect a lot of data; but it will also increase the load on your CPU, and in some cases the rest of your generation infrastructure may not allow multiple simultaneous instances at all. However, in any case you must ensure that the ports on which your server and its instances listen to connections p to p+k must be accessible from the outside. It may be necessary to get your sysadmin to open some holes in your firewall for this.

Setting up the database

While it is by no means necessary, it may be useful for you to attach your local Matchmaker to a database to store previous game runs more permanently. If you want to do this, your database has to conform to a certain structure. We refer you to for details.

Running a GIVE server under Windows

The above instructions were all written with a Unix-based operating system (such as Linux or MacOS) in mind. There is no reason why you shouldn't be able to run a GIVE server on Windows. However, the scripts in the scripts directory are all Bash scripts, which you will only be able to use on Windows if you run them from Cygwin. Alternatively, you can write Windows batch files to achieve the same functionality.

Implementing your own NLG server

At this point, you know how to set up and run your own copy of the GIVE system, using one of the demo servers from the NLG server distribution. We will now explain how to build your own NLG system, and then go into some details about what classes and methods you can use from within your NLG system in order to access the world state and the plan. You can find more details about any of these classes in the Javadocs.

The basics

In order to implement your own NLG system, you need to write a Java class which is a subclass of NlgSystem. This is an abstract base class with a number of abstract callback methods that you need to implement. These callback methods are explained in more detail below. Conversely, you can call NlgSystem's send method to send a string to be displayed in the client at any time you like. send and the other methods you can use from within your NLG system are also described below.

Your NLG system is inserted into an NLG server by specifying the full name of your NlgSystem-derived class in the NLG server configuration file, as described above. You should then build a Jar file that contains your entire NLG system including the NlgSystem-derived class and all the classes that make up the GIVE-2 NLG server infrastructure. This sounds complicated, but if you use Maven to build your NLG server, we have simplified the process in the following way.

A Maven project is generally defined by a POM file, typically with the name pom.xml. If you look at the POM file for the Example NLG System from the NLG server distribution, you will find that it has a section at the top that looks as follows:

01.<!-- edit these to suit your project -->
03. <groupId>edu.union</groupId>
04. <artifactId>give2-example-nlgserver</artifactId>
05. <name>give2-example-nlgserver</name>
06. <version>1.0.1</version>
08. <!-- if you don't care about Maven, you shouldn't have to make any changes below this line -->

To compile your own NLG system, you can simply copy this POM file into your own project directory and adjust the group ID, artifact ID, name, and version as needed. You can then run mvn install assembly:assembly to compile an executable Jar file that you can run with "java -jar blah.jar your-nlg-config.xml". This Jar file will be in the target subdirectory, and have a name consisting of your group and artifact ID, project version, and the prefix "-jar-with-dependencies.jar".

The standard POM file should take care of all dependencies into the GIVE-2 code, and download GIVE packages, automatically. In particular, every time we release a new version of the GIVE server infrastructure, Maven will download the new version and recompile your system against it. In general, you don't need to make any changes below the line that starts "if you don't care about Maven". However, you may want to add your own project dependencies or Maven plug-in configuration options below that line as needed. Feel free to do this, as long as the give2-* dependencies stay intact.

Notice that Maven requires you to follow a certain convention about the structure of your source code directories. This is explained here. If you cannot use Maven for some reason, please contact us, and we can work out a way to get you started anyway.

Access to the world state

States of the virtual world are represented by objects of the class Your NLG system class inherits a method getWorld() which can be used to retrieve the current world object. This object is automatically updated when the user moves around or manipulates objects. You can query the world state by calling methods of World. In particular, you can retrieve all objects in the world by calling getUniverse; the type (button, wall, room, etc.), state, position, orientation, and attributes of each object by calling getType, getState, getPosition, getOrientation, and getAttributes; the player's position and orientation by calling getPlayerPosition and getPlayerOrientation; and the complete list of all known true atoms by calling getTrueAtoms. See the description of the GIVE game worlds for details.

Notice that although World has methods that change the world state, it makes no sense for you to call them. Such changes are not propagated back to the client, and would thus destroy the synchronization between the world state that the client (and thus the user) and your system assume.

Positions and orientations are represented in the world state as 3D floating-point coordinates. The position represents the center of the object. The orientation of an object, for objects where this makes sense, is the normal vector of the side of the object that faces the player when the player interacts with the object. For wall-like objects (doors, walls, safes), these are the sides that are visible to the player. To give you a sense of the dimensions of the world, the width of a door is 1.

The discretizer

You can use the 3D coordinates in the current world state as you see fit. However, a planner will typically not be able to process 3D coordinates; it only understands discrete logical constants. To bridge this gap, your class can access a discretizer by calling getDiscretizer(). The purpose of a discretizer is to translate back and forth between 3D coordinates and discrete names for locations and orientations.

For the current challenge, we have provided the class RegionDiscretizer. This discretizer splits the world into discrete regions according to the following rules:

  1. All regions are rectangles whose sides are parallel to the X and Z axis.
  2. The inside of each room consists of one or more initial regions. These regions are built in such a way that it is guaranteed that if region r1 is adjacent to the region r2 in the same room, then any position in r2 is visible from any position in r1. Rectangular rooms consists of a single region; more complex room shapes may be split up into several regions.
  3. For each door, there are initial regions on both sides of the door.
  4. For each button, trophy, and safe, there are initial regions representing the positions that are close enough to the object to interact with it.
  5. The regions that the discretizer computes are the intersections of the initial regions. That is, if there are two buttons b1 and b2 in a room r, there will be regions reg_r_b1, reg_r_b2, reg_r_b1_b1, and reg_r, representing the positions that are close to b1 but not b2, close to b2 but not b1, close to both, and close to neither.

You can ask the discretizer object to map 3D positions into region names. You can also ask it about adjacency relations between regions, and retrieve a RegionInfo object that gives you information about a certain region. See the Javadocs for details.

The callback methods

Your NLG system reacts to events in the virtual world by implementing several callback methods. If you look at the API of the class NlgSystem, you will see that this is an abstract class with five abstract methods. You must implement each of these methods. They will then be called by the rest of the GIVE server infrastructure at appropriate moments.

In detail, the role of these methods is as follows:

  1. When the connection between the client and the server is established, the server calls the method connectionEstablished. At this point, the world, the discretizer, and the networking system are initialized, and you can use them e.g. to initialize your own data structures. The connectionEstablished method receives an object of type QuestionnaireData as its argument. This object represents the demographic data that the user has specified on the questionnaire that was displayed before the 3D window was started up. If you like, you can use the information in this object to tailor your instructions to the user.
  2. Whenever the user performs an action, the server calls the method handleAction. Here "action" is defined as a mouse click on an object -- pushing a button, picking up the trophy, etc. The server passes the action instance that was executed and a list of all literals that are now true in the world and that weren't true before the action as arguments. At the time that handleAction gets called, the world field has already been updated to reflect the changes in the world.
  3. At regular time intervals (currently, once per 200 ms, although this may change), the client sends a status message to the server. Whenever this happens, the server calls the method handleStatusInformation. The arguments of this call are the current player position, the current player orientation, and the list of all objects that the user can currently see. You can look these objects up in the world state to find out more about them. Note that the computation of the visible objects is not completely precise and may over- or undergenerate slightly.
  4. The client allows the user to signal that they don't know what they are supposed to do, by pressing the H key. Whenever they do this, the method handleDidNotUnderstand gets called. This gives you the chance to repeat or rephrase your previous instruction.
  5. Finally, when the network connection between the client and the NLG server terminates (because the user successfully completed the task, failed to complete the task, cancelled the task, the client crashed, an exception was thrown in the server, or the network connection timed out), the method connectionDisconnected is called. This gives you a chance to clean up any resources that your system may have allocated, before the server instance is terminated. Notice that you can't send any further messages to the client at this point, as the network connection has been closed.

In all these methods except connectionDisconnected, you can throw an NlgServerException if an error occurs in your code. When you throw an exception, it is logged in the database, and the NLG server instance is then shut down in a controlled fashion.

Initialization code for your class can go either into connectionEstablished or into the constructor. You must define a public constructor that takes exactly three parameters, of types NlgServerInstance, World, and Discretizer. It must then pass these arguments on to the super constructor with the same parameters. You are welcome to do more in your constructor. However, you should be aware that the world and discretizer fields are not yet initialized at this point, and there is no network connection to the client yet. This means that any initialization code that depends on the virtual world belongs into connectionEstablished instead.

A new server instance object is created every time the NLG server serves a new client instance. This leads to the creation of a new object of your NLG system class, and therefore to a call to the constructor. In addition, the constructor is also called once in the very beginning, when the NLG server starts up; this call is distinguished by the fact that all arguments are null. If you want to perform expensive initialization code for your NLG system -- e.g., reading and preprocessing a grammar --, you can do this in this initial call to the constructor, and store your results in static fields which can then be accessed by each new instance of your class.

Utility methods you can use

There are several utility methods which you can use in your NLG system.

  • To send a string s to the client, call send(s). This string will then be displayed to the user.
  • To log a message m in the database, call log(m). This message will then be stored in the database. After the evaluation phase, we will make the complete database logs for your system available to you; so feel free to use log to track the internal state of your system as much as you like.
  • To measure the CPU time that your system spends in certain modules, call logStartComputation and logEndComputation. You can give the computation a label, such as "RE generation" or "textplanning"; when you send the next message to the client, all runtimes with the same label will be summed up and added to the log. You can then later analyze how much runtime your system spent in each module. Notice that we measure CPU time, so the measurements should be independent of other processes that run on your machine; but of course, running the system on a faster machine will lead to lower runtimes.
  • Whenever your system attempts to refer to a certain object, you should call logReferenceAttempt and pass the name of the object as an argument. This will allow us to automatically evaluate the ratio of the referring expressions you generated that the user resolved correctly. We are currently discussing whether we can log more about successful attempts to convey information to the user, and may add another method to this effect.
  • To compute a sequence of actions that will achieve a given goal, you can call getPlan. Because the planner doesn't understand about 3D positions and orientations, the world state and the goal will be automatically discretized when the input for the planner is computed, and the action instances that getPlan returns will talk about the symbolic names that the discretizer assigned to positions and orientations, rather than the positions and orientations themselves.


The user moves around in and interacts with the virtual environment by means of actions. Whenever they execute an action by clicking on some object, such as pushing a button or picking up the trophy, the handleAction method of your NLG system gets called, as described above; it receives an instance of the action header as its first argument. In this instance, any parameters that refer to the positions of objects or the player in the world are instantiated with the exact 3D coordinates of this object, as an object of class Position. For instance, pushing a button is represented by the manipulate action. If the player clicks on the button b1, changing its state from "off" to "on" while the player's position is (1,0,3.5), then the handleAction method will be passed the term manipulate(b1,off,on,(1,0,3.5)) as its first argument.

The planner uses the same actions to compute a plan that the user should execute to achieve the goal. However, there are two important differences between the actions in the planner and the actions the user executes in the client. First, as we have already described above, the planner cannot deal directly with continuous 3D positions; it has to run a discretizer to split the map into discrete, disjoint rectangular regions, and it will only reason with these rectangular regions instead of the 3D positions. In the example above, assume that the position (1,0,3.5) is in the region with name "reg_room1_b1". Then the planner would compute a plan that contains the action instance manipulate(b1,off,on,reg_room1_b1). You can use the discretizer to map 3D positions to region names at any time.

The second difference is that the planner uses an action move to represent that the player moved from one region into another. This action has no analogue in the action the player actually performs in the 3D client: From the player's perspective, they can simply walk around in the world, and from the NLG system's perspective, player movements do not trigger handleAction calls. The way to find out about player movements in the NLG system is to inspect the current player position from within calls to handleStatusInformation; if you like, you can again discretize the player position to a region name and compare whether the player has entered the region that the plan specified.

The full list of actions is described as part of the specification of GIVE worlds.

Efficiency considerations

Your NLG system will operate under rather stringent performance conditions. On the one hand, you can't take too much time between deciding that you want to say something and actually sending the message, because the user may turn or walk away while you are still generating. On the other hand, if we are as successful as we hope in recruiting experimental subjects online, you can expect many, perhaps simultaneous connections that will place a high computational load on your machine. You should take these factors into account while you're implementing your system.

One particular issue you should consider is splitting up your computation into separate threads. At the moment, all callback methods of your NLG system run in the same thread. This means that while your system is still busy reacting to one callback method, it won't process the next. If you have an expensive computation -- which, say, takes more than a second --, you should consider executing this computation in a new thread. The relevant methods in NlgSystem are thread-safe, so you shouldn't have to worry about concurrent access to send messing up the internal state of the server. However, you must be aware that exceptions that you throw in the new thread are not caught and logged by the GIVE infrastructure. Furthermore, runtime measurements are per-thread, i.e. in order to obtain meaningful runtime measurements, you must call logStartComputation and logEndComputation in the same thread that executes the computation.

When the server wants to shut down a server instance for any reason (e.g., because the user cancelled the client), it interrupts the message handling thread in which your callback methods run and releases the server instance. (See the documentation on Java threads to learn about interrupts.) At this point, new connections are accepted. This means that you want to check for isInterrupted frequently, and exit your callback method quickly if you discover the interrupt; otherwise a thread from an old instance of your NLG system might still continue running while the server has already spawned the next instance. It is your responsibility to react to interrupts in a correct and timely fashion.

One final caveat: Don't call System.exit in your code! This will terminate the entire server, and not just your instance. The correct way to terminate your instance is to throw an NlgServerException. This will cause the NLG server to shut your instance down in a controlled way, and will also log the error message you specify in the exception in the database for later analysis.

Using languages other than Java

The GIVE infrastructure assumes that your NLG system can be encapsulated in a Java class. This means that at least some part of your system must be implemented in a language that compiles to Java bytecode, such as Java, Scala, or Groovy. We are aware that many participants have extensive NLG software which they want to use although it isn't written in Java. In such cases, we recommend you implement a thin Java (or Scala, or Groovy) wrapper that plugs into the GIVE system as intended and then interfaces with the rest of your generation system on the backend, e.g. through a socket. In this case, it may be convenient for you to restrict the number of simultaneous connections to one.