[disclaimer] This web site was prepared for NTCIR-11 RECIPE and the information on this site is kept for future reference. The task participants and non-participants who have obtained the permission to use the research data can continue to use the provided data. Please note that the distribution period of NTCIR-11 RECIPE English task data has been already ended. Thank you for your understanding.
NTCIR-11 Cooking Recipe Search Task
1. Task name and short name
The task name is Cooking Recipe Search. The short name is RecipeSearch.
2. Task type (core challenge or pilot)
NTCIR-11 RecipeSearch is an additional pilot task.
3. Task description
Information access tasks involving food have traditionally focused on locating
or ranking relevant restaurants given a user’s information need. However, home
cooking remains a fundamental method for acquiring a meal. In this pilot task, we
propose exploring the information access tasks associated with cooking recipes.
Japanese/English recipe search collections will be constructed in this task.
In RecipeSearch, we selected the following two sub-tasks to study.
 The first subtask, ad-hoc search, considers the scenario where a user searches
for a recipe using a natural language question.
This includes straightforward queries such as 'curry rice' as well as more
complicated queries such as 'curry rice without gluten'.
We expect these constraints to be common in real world queries due to dietary
restrictions and limited availability of some ingredients.
 The second subtask, recipe pairing, considers the scenario where a user
searches for a complementary recipe (side dish) to a query recipe (main dish).
For example, a user may be interested in 'recipes complementing curry rice',
such as a soup, salad, or dessert. This sub-task, although similar to IR tasks
such as diversification and contextual suggestion, focuses on the unique properties
of cooking and eating.
Dataset (corpus & queries)
Rakuten Recipe contains 440,000 Japanese recipes. Yummly Recipe Data contains
100,000 English recipes. For subtask1 (ad-hoc search), 500 Japanese queries and
500 English queries are used for the evaluation. Each of the queries is associated
with a sample relevant recipe. Each of the queries contain simple keywords.
For subtask2 (recipe pairing), 100 Japanese queries and 100 English queries are used
for the evaluation. Each of the query contains a recipe of main-dish (meat, pasta, etc.)
as context information. A recipe of side-dish (a soup, salad, or dessert) is information
need that users want to search for. For more details, see corpus & queries.
Judgements (relevance assessment)
Participants are expected to generate ranked recipe lists (called as "runs"), and
submit them to TOs. Please note that participants in this task is requested to give
manual judgments to develop official judgements data in the RecipeSearch task.
Participants may employ students to do this process.
4. Schedule (revised on 13 September)
 Apr, 2014 Website open by TOs
 May, 2014 Task participants data agreement with the data provider
 May-June, 2014 (2014/05/01-2014/06/29) Query development by TOs
 June, 2014 Task participants registration open by NII/NTCIR
 June, 2014 (2014/06/30) Search topics provided by TOs
 July, 2014 (2014/07/01-2014/07/31) Formal run
 July, 2014 (2014/07/31) Task participants registration due
 Aug, 2014 (2014/08/01-2014/08/28) Relevance assessment & Run submission
 Sep, 2014 (2014/09/01) Evaluation results release by TOs
 Sep, 2014 (2014/09/01) Early draft Task overview release by TOs
 Sep, 2014 (2014/09/22) Task participants daft abstracts due (submit to TOs)
 Oct, 2014 (2014/10/15) Task overview & Task participants camera
ready due (for Data Provider's review)
Nov, 2014 (2014/11/01) All camera ready abstracts due
After  in the above schedule, important dates are same as the other tasks.
5. Important notice
According to the agreements with the data providers, participants must be researchers in universities or public institutions, and the usage of the data is strictly limited to research purposes only. Note that this limitation is not because of intention of NII or NTCIR, but because of the respect for the agreements with the data providers.
6. Contact information of the task
Communications with TOs or inquiries to TOs should be sent to the following e-mail address:
ntcir11.recipesearch [at] daisy.cs.gunma-u.ac.jp
7. Task Organizers
Gunma University, Japan
Institute of Nutrition Sciences,
Kagawa Nutrition University, Japan
Documents and experiment data for the Recipe Search task were provided
by the following data providers. We express our gratitude to the data providers.
[Food names for experiments]
STANDARD TABLES OF FOOD COMPOSITION IN JAPAN - 2010 -
Report of the Subdivision on Resources,
The Council for Science and Technology,
Ministry of Education, Culture, Sports, Science and Technology, JAPAN
The Rakuten Data provided by Rakuten, Inc.
Yummly Recipe Data v1 provided by Yummly
Assessments and annotations for data creation in the Recipe Search task
were performed by the participant groups in the Recipe Search task.
We are grateful for their hard work.