Generation Challenge

Call for Expressions of Interest: the GRUVE challenge (**please distribute**)

We solicit your participation in the GRUVE challenge. GRUVE stands for "Giving Route instructions in Uncertain Virtual Environments". We see this challenge as a successor to the Generation Challenge GIVE (Giving Instructions in Virtual Environments) challenge, where the objective was for you, as a participating team, to build an NLG system that generates instructions for a user to navigate inside a virtual building, and manipulate buttons in order to obtain a hidden trophy. In this challenge (GRUVE), you will be building an NLG module which will assist a user navigating through the streets of a city on Google Streetview. The challenge here is to generate instructions in the context of uncertain user positioning information and to adapt them to a user's experience in the neighborhood (see below). Please read our ACL 2012 paper on GRUVE for more details http://aclweb.org/anthology-new/P/P12/P12-3009.pdf

How will users interact with the system?

Users will interact with the system as they play a game to help a character (e.g. a pirate) find an object (e.g. a treasure chest). In the pirate scenario, the treasure chest is hidden in one of the streets in the neighbourhood. In order to find and open this chest, the user must obtain the key for the chest. This puts the user on a quest to find the key and the chest. This requires the users to navigate to destinations in search of clues, keys and other objects that might be useful in their quest. Users will interact with the system in order to get navigation instructions to destinations of their choice by using drop down lists and buttons to communicate dialogue actions of the user to the system. The system in turn responds with navigation instructions using speech, by turning the utterance generated by the NLG module into speech using a TTS module.

Research Challenges

1. Generating navigation instructions: Given the user location and goal (i.e. a destination street name), the system should generate navigation instructions for the user. Our dialogue manager module will present your NLG module with the route plan for the user to get to the destination from their current location at every decision point (e.g. street junction). The NLG module will be required to convert this route plan into natural language instructions.

2. Uncertainty in user location: User location reported in terms of his/her latitude and longitude may be erroneous. The uncertainty will be passed to the NLG module by the dialog manager module in terms of an accuracy metric (with range of 1 to 50 meters). For example, an accuracy of 50 meters means that the user is within 50 meters of the given coordinate. In the face of uncertainty, the challenge for the NLG module is to generate more robust instructions that users can use even if their locations are uncertain.

3. Modelling the user's spatial knowledge: If users return to play the same game or some other game in the same geographic region, NLG modules could take that into account and generate instructions that are adapted to the user's knowledge of the neighbourhood. Each user will be uniquely identified, and the NLG module will be able to query the user's past navigation experience in the neighbourhood from the user model as recorded by the dialogue manager. This information can be used to adapt the instruction generated to the user's navigation traces.

Evaluation

All participating NLG modules will be hosted on the GRUVE web server at Heriot -Watt university and will be evaluated by web users recruited via crowdsourcing.

Proposed timeline

1. Download GRUVE toolkit and set up (Dec 2012)

2. Development of NLG modules (Jan - Apr 2013)

3. Evaluating NLG modules (May 2013)

4. Analysis of results (June 2013)

5. Reporting results at Generation Challenges at ENLG 2013 (Aug 2013)

What to do next?

Please write to us (srinivasancj@gmail.com) if you are interested in participating in this shared task. We will provide you with the GRUVE toolkit to get started and build your own NLG system!