NTCIR-11 Cooking Recipe Search Task
1. Task name and short name The task name is Cooking Recipe Search. The short name is RecipeSearch. 2. Task type (core challenge or pilot) NTCIR-11 RecipeSearch is an additional pilot task. 3. Task description Information access tasks involving food have traditionally focused on locating or ranking relevant restaurants given a user’s information need. However, home cooking remains a fundamental method for acquiring a meal. In this pilot task, we propose exploring the information access tasks associated with cooking recipes. Japanese/English recipe search collections will be constructed in this task. In RecipeSearch, we selected the following two sub-tasks to study. [1] The first subtask, ad-hoc search, considers the scenario where a user searches for a recipe using a natural language question. This includes straightforward queries such as 'curry rice' as well as more complicated queries such as 'curry rice without gluten'. We expect these constraints to be common in real world queries due to dietary restrictions and limited availability of some ingredients. [2] The second subtask, recipe pairing, considers the scenario where a user searches for a complementary recipe (side dish) to a query recipe (main dish). For example, a user may be interested in 'recipes complementing curry rice', such as a soup, salad, or dessert. This sub-task, although similar to IR tasks such as diversification and contextual suggestion, focuses on the unique properties of cooking and eating. Dataset (corpus & queries) Rakuten Recipe contains 440,000 Japanese recipes. Yummly Recipe Data contains 100,000 English recipes. For subtask1 (ad-hoc search), 500 Japanese queries and 500 English queries are used for the evaluation. Each of the queries is associated with a sample relevant recipe. Each of the queries contain simple keywords. For subtask2 (recipe pairing), 100 Japanese queries and 100 English queries are used
for the evaluation. Each of the query contains a recipe of main-dish (meat, pasta, etc.) as context information. A recipe of side-dish (a soup, salad, or dessert) is information need that users want to search for. For more details, see corpus & queries. Judgements (relevance assessment) Participants are expected to generate ranked recipe lists (called as "runs"), and submit them to TOs. Please note that participants in this task is requested to give manual judgments to develop official judgements data in the RecipeSearch task. Participants may employ students to do this process. 4. Schedule (revised on 13 September)
[1] Apr, 2014
Website open by TOs
[2] May, 2014 Task participants data agreement with the data provider
[3] May-June, 2014 (2014/05/01-2014/06/29) Query development by TOs
[4] June, 2014 Task participants registration open by NII/NTCIR [5] June, 2014 (2014/06/30) Search topics provided by TOs[6] July, 2014 (2014/07/01-2014/07/31) Formal run
[7] July, 2014 (2014/07/31) Task participants registration due
[8] Aug, 2014 (2014/08/01-2014/08/28) Relevance assessment & Run submission [9] Sep, 2014 (2014/09/01) Evaluation results release by TOs [10] Sep, 2014 (2014/09/01) Early draft Task overview release by TOs [11] Sep, 2014 (2014/09/22) Task participants daft abstracts due (submit to TOs) [12] Oct, 2014 (2014/10/15) Task overview & Task participants camera ready due (for Data Provider's review) Nov, 2014 (2014/11/01) All camera ready abstracts due After [12] in the above schedule, important dates are same as the other tasks. See) http://research.nii.ac.jp/ntcir/ntcir-11/dates.html 5. Important notice
According to the agreements with the data providers, participants must be researchers in universities or public institutions, and the usage of the data is strictly limited to research purposes only. Note that this limitation is not because of intention of NII or NTCIR, but because of the respect for the agreements with the data providers.
6. Contact information of the task Communications with TOs or inquiries to TOs should be sent to the following e-mail address: ntcir11.recipesearch [at] daisy.cs.gunma-u.ac.jp 7. Task Organizers Michiko Yasukawa Gunma University, Japan Fernando Diaz Microsoft, USA Gregory Druck Yummly, USA Nobu Tsukada Institute of Nutrition Sciences, Kagawa Nutrition University, Japan 8. Acknowledgement Documents and experiment data for the Recipe Search task were provided by the following data providers. We express our gratitude to the data providers. [Food names for experiments] STANDARD TABLES OF FOOD COMPOSITION IN JAPAN - 2010 - Report of the Subdivision on Resources, The Council for Science and Technology, Ministry of Education, Culture, Sports, Science and Technology, JAPAN [Japanese search] The Rakuten Data provided by Rakuten, Inc. [English search] Yummly Recipe Data v1 provided by Yummly Assessments and annotations for data creation in the Recipe Search task were performed by the participant groups in the Recipe Search task. We are grateful for their hard work. |