We are excited to announce a shared task on Explanation Regeneration! The resulting papers will be peer-reviewed by participating teams, and accepted system descriptions will be presented along with the main workshop papers.

Brief Overview

Multi-hop inference is the task of combining more than one piece of information to solve an inference task, such as question answering. This can take many forms, from combining free-text sentences read from books or the web, to combining linked facts from a structured knowledge base. The Shared Task on Explanation Regeneration asks participants to develop methods to reconstruct gold explanations for elementary science questions, using a new corpus of gold explanations that provides supervision and instrumentation for this multi-hop inference task. Each explanation is represented as an "explanation graph", a set of atomic facts (between 1 and 16 per explanation, drawn from a knowledge base of 5,000 facts) that, together, form a detailed explanation for the reasoning required to answer and explain the resoning behind a question. Linking these facts to achieve strong performance at rebuilding the gold explanation graphs will require methods to perform multi-hop inference. The explanations include both core scientific facts as well as detailed world knowledge, allowing this task to appeal to those interested in both multi-hop reasoning and common-sense inference.

Please follow the links for details of the task and submission instructions:

Shared Task Overview: https://github.com/umanlp/tg2019task

Competition on CodaLab: https://competitions.codalab.org/competitions/23047

Task

Participating systems are asked to perform an explanation reconstruction task, a stepping-stone task towards general multi-hop inference on large graphs. The task is as follows: Given a question and known correct answer, build a system that reconstructs the gold explanation. For ease of evaluation (and to encourage a variety of methods, not only those involving graph-based inference), the task is framed as a ranking task where for a given question, one must selectively rank facts in the gold explanation higher than facts not present in the gold explanation.

Important Dates

• 13-05-2019: Example (trial) data release
• 17-05-2019: Training data release
• 12-07-2019: Test data release; Evaluation start
• 09-08-2019: Evaluation end
• 23-08-2019: System description paper deadline
• 09-09-2019: Deadline for reviews of system description papers
• 16-09-2019: Author notifications
• 30-09-2019: Camera-ready description paper deadline
• 03-11-2019/04-11-2019: TextGraphs-13 workshop

The dates are specified in the following format: day-month-year.

Submission

Please submit your solutions via CodaLab: https://competitions.codalab.org/competitions/20150

Citation

• Jansen P. and Ustalov D. TextGraphs 2019 Shared Task on Multi-Hop Inference for Explanation Regeneration. Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13). Hong Kong: Association for Computational Linguistics, 2019, pp. 63–77.

@inproceedings{Jansen:19,

  author    = {Jansen, Peter and Ustalov, Dmitry},

  title     = {{TextGraphs~2019 Shared Task on Multi-Hop Inference for Explanation Regeneration}},

  booktitle = {Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13)},

  year      = {2019},

  pages     = {63--77},

  url       = {https://www.aclweb.org/anthology/D19-5309},

  isbn      = {978-1-950737-86-4},

  address   = {Hong Kong},

  publisher = {Association for Computational Linguistics},

  language  = {english},

}

Contacts

We welcome questions and answers on the shared task on GitHub: https://github.com/umanlp/tg2019task/issues.