We are excited to announce a shared task on Explanation Regeneration! The resulting papers will be peer-reviewed by participating teams, and accepted system descriptions will be presented along with the main workshop papers.
Brief OverviewMulti-hop inference is the task of combining more than one piece of information to solve an inference task, such as question answering. This can take many forms, from combining free-text sentences read from books or the web, to combining linked facts from a structured knowledge base. The Shared Task on Explanation Regeneration asks participants to develop methods to reconstruct gold explanations for elementary science questions, using a new corpus of gold explanations that provides supervision and instrumentation for this multi-hop inference task. Each explanation is represented as an "explanation graph", a set of atomic facts (between 1 and 16 per explanation, drawn from a knowledge base of 5,000 facts) that, together, form a detailed explanation for the reasoning required to answer and explain the resoning behind a question. Linking these facts to achieve strong performance at rebuilding the gold explanation graphs will require methods to perform multi-hop inference. The explanations include both core scientific facts as well as detailed world knowledge, allowing this task to appeal to those interested in both multi-hop reasoning and common-sense inference.
Please follow the links for details of the task and submission instructions:
Shared Task Overview: https://github.com/umanlp/tg2019task
Competition on CodaLab: https://competitions.codalab.org/competitions/23047
TaskParticipating systems are asked to perform an explanation reconstruction task, a stepping-stone task towards general multi-hop inference on large graphs. The task is as follows: Given a question and known correct answer, build a system that reconstructs the gold explanation. For ease of evaluation (and to encourage a variety of methods, not only those involving graph-based inference), the task is framed as a ranking task where for a given question, one must selectively rank facts in the gold explanation higher than facts not present in the gold explanation.
Important DatesThe dates are specified in the following format: day-month-year.
SubmissionPlease submit your solutions via CodaLab: https://competitions.codalab.org/competitions/20150
Citation@inproceedings{Jansen:19, author = {Jansen, Peter and Ustalov, Dmitry}, title = {{TextGraphs~2019 Shared Task on Multi-Hop Inference for Explanation Regeneration}}, booktitle = {Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13)}, year = {2019}, pages = {63--77}, url = {https://www.aclweb.org/anthology/D19-5309}, isbn = {978-1-950737-86-4}, address = {Hong Kong}, publisher = {Association for Computational Linguistics}, language = {english},}ContactsWe welcome questions and answers on the shared task on GitHub: https://github.com/umanlp/tg2019task/issues.