Calls: Demo, Data

Call for Dataset and Demos Showing State-of-Art in Reasoning and Learning for Human-Machine Dialogues

Partially sponsored by AI Journal

A key part of the workshop will be an hour long session dedicated to demonstrations and datasets that illustrate the state-of-art in the field. In that spirit, we invite (a) conversation datasets where both reasoning and learning are needed to excel, and (b) demonstrations of conversation agents (chatbots) in a given domain and . Using sponsorship from AI Journal, we will offer two awards/ support grants for each track. In addition, we will support two students / early researchers.

Prizes and support

  • Competitive prize for demo (chatbot) in given usecase: 1st prize - $700, 2nd prize - $500
  • Competitive recognition for data contributions (two) - $ 500 each
  • Student support (two) - $ 500 each

Submission Site: https://sites.google.com/view/deep-dial2020

Deadline:

  • Intent to participate with team and abstract, Nov 15th, 2019
  • Final Submission: Jan 15th, 2020

Dataset Call: Conversation Response With Reasoning and Learning


Motivation

Most public dialog datasets today [1] are geared towards learning of policy for generating response. We are interested in datasets where generating response requires the ability to learn and reason with data and domain / universal concepts (like space and time) [2].

Submission: A team may submit a 2-page with information about

  1. Team name and members. Identify student members and also indicate if support is needed for student to attend DEEP-DIAL 20 workshop
  2. Information about dataset: how it was assembled, description and links.
  3. Illustration of 2 conversation use-cases showing need for reasoning and learning
  4. A chatbot able to converse in the domain (preferred)

Clarifications and Contact: If you have any questions, please contact: biplavs AT us.ibm.com

References

  1. A Survey of Available Corpora for Building Data-Driven Dialogue Systems, https://breakend.github.io/DialogDatasets/
  2. Foundations of Collaborative Task-Oriented Dialogue: What’s in a Slot?, Philip Cohen, https://www.aclweb.org/anthology/W19-5924/, 2019

Demonstration Call: Navigate New York City via Conversation and Open Data

Motivation

AAAI2020 will be hosted in New York City (NYC). In that spirit, we appeal to community’s competitive spirit to build a chat bot that can help AAAI2020 attendees navigate NYC.

Example Scenario:

Data @NYC (data.cityofnewyork.us) is a great resource of readily available open data around NYC subway stations and lines, MTA (Metropolitan Transportation Authority of the State of New York) subways, buses, commuter rail, bridges, and tunnels, Points Of Interest points, museums, galleries, theaters and open spaces. We expect submitted chatbots to be able to seamlessly answer questions regarding any of these resources in the context of conversation.

Examples: How do I got from location A to location B?, What are other point-of-interest around location a?, How long point-of-interest A is open for?, What's the contact information for museum b? etc. See two detailed use-cases below.

Data Sources:

  1. Subway Lines (https://data.cityofnewyork.us/Transportation/Subway-Lines/3qz8-muuu)
  2. Subway Stations (https://data.cityofnewyork.us/Transportation/Subway-Stations/arq3-7z49)
  3. Point-of Interest (https://data.cityofnewyork.us/City-Government/Points-Of-Interest/rxuy-2muj)
  4. Museums and Galleries (https://data.cityofnewyork.us/Recreation/Museums-and-galleries/kcrm-j9hh)
  5. Open spaces and parks (https://data.cityofnewyork.us/Recreation/Open-Space-Parks-/g84h-jbjm)
  6. MTA Data (https://data.cityofnewyork.us/Transportation/MTA-Data/mmu8-8w8b)
  7. OpenMap (https://www.openstreetmap.org)

=============================== USE CASE 1 =======================================

** Theme: - Exploring a city space

User: Hello, I am currently at Central Park north and was looking for things to do around here. 
System: Sure. I can assist you with that. How about at The Metropolitan Museum of Art (Based on entry in Museums DB.) it's located at E. 82nd St.. It's a mile walk from where you are. Do you want me to route you?
User: Will it be open when I get there. 
System: It should be open. (Based on entry in Museums DB.), but if you want to check it by yourself I can provide you with the website to it or even contact number. 
User: I think it's better if I call and check. 
System: The Metropolitan Museum of Art can be reached at 212-535-7710. 
User: Thanks. Can you also show me how to get there?
System: Yes. Let me pull up instructions from OpenMap. 
User: Great. Thanks.

Knowledge sources: Open Spaces and Parks, Point-of-interest, Museums and Galleries.

Navigation/Map source: OpenMap

=============================== USE CASE 2 =======================================

** Theme: Navigating in a city

User: I came to see the world trade center but my friend is waiting at time square. How do I get there?
System: You can get there by subway. Do you want me to give you subway instructions?
User: yes please. 
System: Walk to Chambers street station from the world trade center. Then take a "A" train going toward "Inwood". Get down at "42 St - Port Authority Bus Terminal" stop and from there it's a short walk to time square.
User: Thanks.

Knowledge sources: Point-of-interest, Subway Stations, Subway Lines, MTA

Navigation/Map source: OpenMap

=================================================================================

Review Criteria

  1. Ability of chatbot to answer queries related to subject matter (e.g., points of interest, train stations in preferred scenario)
  2. Ability of chatbot to handle users of different backgrounds leading to dialogs of different lengths (e.g., exact terms, partial matches, switching intents)
  3. Ability of chatbot to handle multiple turns
  4. Ability to handle abusive and discriminatory language
  5. Response time and error handling
  6. Any special feature. E.g., Ability to handle mixture of languages, showing multi-modal response like maps or graphs when appropriate
  7. Although full deployed solution are valued more, we welcome design/architectural solutions that can lead to full implementation

Submission: A team may submit a 2-page with information about

  1. Team name and members. Identify student members and also indicate if support is needed for student to attend DEEP-DIAL 20 workshop
  2. Information about any additional open dataset used.
  3. A demonstration video of using the chatbot
  4. Link to source code on github (preferred)
  5. URL of actual chatbot that can be tested
  6. Discussion of implemented approach. Reference to a detailed paper is allowed.

Clarifications and Contact: If you have any questions, including confidentiality of the submission, please contact us. Email: kpfadnis AT us.ibm.com and biplavs AT us.ibm.com

Rules

  1. Please note that use of proprietary knowledge/data source without organizers permission is prohibited. If you think a source is valuable, let organizers know. As long as all participants are aware of data and can access the source with sufficient notice, new sources will be permitted.
  2. Chatbot should be available publicly for demonstration for at least 3 months. For example, hosted on any cloud platform.
  3. Source code should be made available on Github.
  4. Data used by chatbot should be open and hence downloadable for free.