[disclaimer] This web site was prepared for NTCIR-11 RECIPE and the information on this site is kept for future reference. The task participants and non-participants who have obtained the permission to use the research data can continue to use the provided data. Please note that the distribution period of NTCIR-11 RECIPE English task data has been already ended. Thank you for your understanding.

NTCIR-11 Cooking Recipe Search Task

1. Task name and short name

The task name is Cooking Recipe Search. The short name is RecipeSearch.

2. Task type (core challenge or pilot)

NTCIR-11 RecipeSearch is an additional pilot task.

3. Task description

Information access tasks involving food have traditionally focused on locating

or ranking relevant restaurants given a user’s information need. However, home

cooking remains a fundamental method for acquiring a meal. In this pilot task, we

propose exploring the information access tasks associated with cooking recipes.

Japanese/English recipe search collections will be constructed in this task.

In RecipeSearch, we selected the following two sub-tasks to study.

[1] The first subtask, ad-hoc search, considers the scenario where a user searches

for a recipe using a natural language question.

This includes straightforward queries such as 'curry rice' as well as more

complicated queries such as 'curry rice without gluten'.

We expect these constraints to be common in real world queries due to dietary

restrictions and limited availability of some ingredients.

[2] The second subtask, recipe pairing, considers the scenario where a user

searches for a complementary recipe (side dish) to a query recipe (main dish).

For example, a user may be interested in 'recipes complementing curry rice',

such as a soup, salad, or dessert. This sub-task, although similar to IR tasks

such as diversification and contextual suggestion, focuses on the unique properties

of cooking and eating.

Dataset (corpus & queries)

Rakuten Recipe contains 440,000 Japanese recipes. Yummly Recipe Data contains

100,000 English recipes. For subtask1 (ad-hoc search), 500 Japanese queries and

500 English queries are used for the evaluation. Each of the queries is associated

with a sample relevant recipe. Each of the queries contain simple keywords.

For subtask2 (recipe pairing), 100 Japanese queries and 100 English queries are used

for the evaluation. Each of the query contains a recipe of main-dish (meat, pasta, etc.)

as context information. A recipe of side-dish (a soup, salad, or dessert) is information

need that users want to search for. For more details, see corpus & queries.

Judgements (relevance assessment)

Participants are expected to generate ranked recipe lists (called as "runs"), and

submit them to TOs. Please note that participants in this task is requested to give

manual judgments to develop official judgements data in the RecipeSearch task.

Participants may employ students to do this process.

4. Schedule (revised on 13 September)

[1] Apr, 2014 Website open by TOs

[2] May, 2014 Task participants data agreement with the data provider

[3] May-June, 2014 (2014/05/01-2014/06/29) Query development by TOs

[4] June, 2014 Task participants registration open by NII/NTCIR

[5] June, 2014 (2014/06/30) Search topics provided by TOs

[6] July, 2014 (2014/07/01-2014/07/31) Formal run

[7] July, 2014 (2014/07/31) Task participants registration due

[8] Aug, 2014 (2014/08/01-2014/08/28) Relevance assessment & Run submission

[9] Sep, 2014 (2014/09/01) Evaluation results release by TOs

[10] Sep, 2014 (2014/09/01) Early draft Task overview release by TOs

[11] Sep, 2014 (2014/09/22) Task participants daft abstracts due (submit to TOs)

[12] Oct, 2014 (2014/10/15) Task overview & Task participants camera

ready due (for Data Provider's review)

Nov, 2014 (2014/11/01) All camera ready abstracts due

After [12] in the above schedule, important dates are same as the other tasks.


5. Important notice

According to the agreements with the data providers, participants must be researchers in universities or public institutions, and the usage of the data is strictly limited to research purposes only. Note that this limitation is not because of intention of NII or NTCIR, but because of the respect for the agreements with the data providers.

6. Contact information of the task

Communications with TOs or inquiries to TOs should be sent to the following e-mail address:

ntcir11.recipesearch [at]

7. Task Organizers

Michiko Yasukawa

Gunma University, Japan

Fernando Diaz

Microsoft, USA

Gregory Druck

Yummly, USA

Nobu Tsukada

Institute of Nutrition Sciences,

Kagawa Nutrition University, Japan

8. Acknowledgement

Documents and experiment data for the Recipe Search task were provided

by the following data providers. We express our gratitude to the data providers.

[Food names for experiments]


Report of the Subdivision on Resources,

The Council for Science and Technology,

Ministry of Education, Culture, Sports, Science and Technology, JAPAN

[Japanese search]

The Rakuten Data provided by Rakuten, Inc.

[English search]

Yummly Recipe Data v1 provided by Yummly

Assessments and annotations for data creation in the Recipe Search task

were performed by the participant groups in the Recipe Search task.

We are grateful for their hard work.