Home

News

  • Dec 12, 2014: Thank you for your participation and cooperation to NTCIR-11 Conference and RITE-VAL task.
  • Nov 14, 2014: Online Registration is available. Early-bird registration is until Nov 21.
  • Nov 14, 2014: The conference program is available.
  • Nov 7, 2014: Thank you for your submission of the camera-ready paper.
  • Sep 29, 2014: We would appreciate it if you could submit your camera-ready paper, where your official results were updated due to the revised EN-FV and/or JA-FV data, by Nov 1, 2014. You do not have to resubmit your draft paper.
  • Sep 29, 2014: Formal run data with correct labels for EN-FV and JA-FV subtasks are revised.
  • Sep 17, 2014: We are receiving Draft participant paper submission on the EasyChair site after Sep 15. We would like to cite works of all the participating teams in our camera-ready task overview paper that will be released on Oct 15.
  • Sep 12, 2014: Partial task overview paper is released.
  • Aug 29, 2014: Formal run data with correct labels for CS and CT subtasks are released.
  • Aug 26, 2014: PC chairs extended the draft participant paper submission due to Sep 15.
  • Aug 21, 2014: Formal run data with correct labels for EN and JA subtasks are released.
  • Aug 11, 2014: Formal run results for EN subtask are reported to the participants via e-mail. Check your mailbox for it.
  • Aug 9, 2014: Formal run results for CS, CT and JA subtasks are reported to the participants via e-mail. Check your mailbox for it.
  • Aug 9 to 17, 2014: NII will be closed for Japanese 'obon' holidays.
  • Aug 7, 2014: Task organizers decided to extend the evaluation results due only for EN subtask to Aug 10 JST23:59.
  • Aug 5, 2014: RITE-VAL JA System Validation subtask is a binary classification (Y/N) task, not a multi-classification task.
  • Aug 4, 2014: Formal run data for EN and JA subtasks are released.
  • Jul 29, 2014: Task organizers decided to extend the evaluation results due for all subtasks, including CS/CT subtasks, to Aug 7 JST23:59.
  • Jul 25, 2014: Formal run data for CS and CT subtasks are released.
  • May 1 to 6, 2014: NII will be closed for Japanese national holidays.
  • Apr 30, 2014: Dry run data for EN, CS, CT and JA subtasks are released.
  • Apr 22, 2014: User agreement forms for EN, CS, CT and JA subtasks are released now.
  • Jan 20, 2014: RITE-VAL Task organizers welcome new participants after Jan 21, 2014. 
  • Aug 20, 2013: Web Site is opened.

Overview

NTCIR-11 RITE-VAL is an evaluation-based workshop aiming to recognize entailment, paraphrase, and contradiction between sentences, which is a common problem shared widely among researchers of natural language processing and information access. By evaluating participanting systems using large scale test collections of Chinese (simplified and traditional), English and Japanese, we expect to obtain new knowledge, promote development of data and tools, and greatly advance the field of recognizing textual entailment.

Task

Who should participate?

RITE-VAL is a generic task that addresses major Text Understanding needs in various research areas e.g. Information Retrieval, Question Answering, Text Summarization, Opinion Analysis. Researchers in Natural Language Processing, Machine Learning community are also very welcome. Since RITE is a friendly evaluation effort, do not hesitate to try ambitious/exploratory approaches that do not necessarily work. A uniqueness in your method would also be a big contribution to the community. Don't worry about the lack of language resource/tool to participate.

Schedule

Aug 31, 2013

Call for task participation and website due

Sep 2, 2013

Kick-off event in Tokyo

Dec 20, 2013

Task participants registration due

Jan, 2014

Apr, 2014

Document set release, Data release

(Sorry for the delay)

Jan – Jul, 2014

Dry run

Jul 31, 2014

Task participants registration due

Jul 25, 2014 (CT and CS)

Aug 4, 2014 (EN and JA)

Formal run data release and Formal run

Jul 31, 2014 (CT and CS)

Aug 7, 2014 (CT, CS and JA)

Aug 10, 2014 (EN)

Evaluation results due

Aug 1, 2014

Sep 12, 2014

Partial task overview paper released to participants

Sep 1, 2014

Sep 15, 2014

Draft participant paper submission due

Oct 1, 2014

Oct 30, 2014

Task overview paper released to participants

Nov 1, 2014

All camera-ready copy for the Proceedings due

Nov 28, 2014

Poster and Demo Session Request Form Due

Dec 9-12, 2014

NTCIR-11 Conference, NII, Tokyo, Japan