[disclaimer] This web site was prepared for NTCIR-11 RECIPE and the information on this site is kept for future reference. The task participants and non-participants who have obtained the permission to use the research data can continue to use the provided data. Please note that the distribution period of NTCIR-11 RECIPE English task data has been already ended. Thank you for your understanding.
NTCIR-11 Cooking Recipe Search Task
1. Task name and short name
The task name is Cooking Recipe Search. The short name is RecipeSearch.
2. Task type (core challenge or pilot)
NTCIR-11 RecipeSearch is an additional pilot task.
3. Task description
Information access tasks involving food have traditionally focused on locating
or ranking relevant restaurants given a user’s information need. However, home
cooking remains a fundamental method for acquiring a meal. In this pilot task, we
propose exploring the information access tasks associated with cooking recipes.
Japanese/English recipe search collections will be constructed in this task.
In RecipeSearch, we selected the following two sub-tasks to study.
[1] The first subtask, ad-hoc search, considers the scenario where a user searches
for a recipe using a natural language question.
This includes straightforward queries such as 'curry rice' as well as more
complicated queries such as 'curry rice without gluten'.
We expect these constraints to be common in real world queries due to dietary
restrictions and limited availability of some ingredients.
[2] The second subtask, recipe pairing, considers the scenario where a user
searches for a complementary recipe (side dish) to a query recipe (main dish).
For example, a user may be interested in 'recipes complementing curry rice',
such as a soup, salad, or dessert. This sub-task, although similar to IR tasks
such as diversification and contextual suggestion, focuses on the unique properties
of cooking and eating.
Dataset (corpus & queries)
Rakuten Recipe contains 440,000 Japanese recipes. Yummly Recipe Data contains
100,000 English recipes. For subtask1 (ad-hoc search), 500 Japanese queries and
500 English queries are used for the evaluation. Each of the queries is associated
with a sample relevant recipe. Each of the queries contain simple keywords.
For subtask2 (recipe pairing), 100 Japanese queries and 100 English queries are used
for the evaluation. Each of the query contains a recipe of main-dish (meat, pasta, etc.)
as context information. A recipe of side-dish (a soup, salad, or dessert) is information
need that users want to search for. For more details, see corpus & queries.
Judgements (relevance assessment)
Participants are expected to generate ranked recipe lists (called as "runs"), and
submit them to TOs. Please note that participants in this task is requested to give
manual judgments to develop official judgements data in the RecipeSearch task.
Participants may employ students to do this process.
4. Schedule (revised on 13 September)
[1] Apr, 2014 Website open by TOs
[2] May, 2014 Task participants data agreement with the data provider
[3] May-June, 2014 (2014/05/01-2014/06/29) Query development by TOs
[4] June, 2014 Task participants registration open by NII/NTCIR
[5] June, 2014 (2014/06/30) Search topics provided by TOs
[6] July, 2014 (2014/07/01-2014/07/31) Formal run
[7] July, 2014 (2014/07/31) Task participants registration due
[8] Aug, 2014 (2014/08/01-2014/08/28) Relevance assessment & Run submission
[9] Sep, 2014 (2014/09/01) Evaluation results release by TOs
[10] Sep, 2014 (2014/09/01) Early draft Task overview release by TOs
[11] Sep, 2014 (2014/09/22) Task participants daft abstracts due (submit to TOs)
[12] Oct, 2014 (2014/10/15) Task overview & Task participants camera
ready due (for Data Provider's review)
Nov, 2014 (2014/11/01) All camera ready abstracts due
After [12] in the above schedule, important dates are same as the other tasks.
See) http://research.nii.ac.jp/ntcir/ntcir-11/dates.html
5. Important notice
According to the agreements with the data providers, participants must be researchers in universities or public institutions, and the usage of the data is strictly limited to research purposes only. Note that this limitation is not because of intention of NII or NTCIR, but because of the respect for the agreements with the data providers.
6. Contact information of the task
Communications with TOs or inquiries to TOs should be sent to the following e-mail address:
ntcir11.recipesearch [at] daisy.cs.gunma-u.ac.jp
7. Task Organizers
Michiko Yasukawa
Gunma University, Japan
Fernando Diaz
Microsoft, USA
Gregory Druck
Yummly, USA
Nobu Tsukada
Institute of Nutrition Sciences,
Kagawa Nutrition University, Japan
8. Acknowledgement
Documents and experiment data for the Recipe Search task were provided
by the following data providers. We express our gratitude to the data providers.
[Food names for experiments]
STANDARD TABLES OF FOOD COMPOSITION IN JAPAN - 2010 -
Report of the Subdivision on Resources,
The Council for Science and Technology,
Ministry of Education, Culture, Sports, Science and Technology, JAPAN
[Japanese search]
The Rakuten Data provided by Rakuten, Inc.
[English search]
Yummly Recipe Data v1 provided by Yummly
Assessments and annotations for data creation in the Recipe Search task
were performed by the participant groups in the Recipe Search task.
We are grateful for their hard work.