Food and cooking analysis present exciting research and application challenges for modern AI systems, particularly in the context of multimodal data such as images or video. A meal that appears in a food image is a product of a complex progression of cooking stages, often described in the accompanying textual recipe form. In the cooking process, individual ingredients change their physical properties, become combined with other food components, all to produce a final, yet highly variable, appearance of the meal. Recognizing food items or meals on a plate from images or videos, their physical properties such as the amount, nutritional content such as the caloric value, food attributes such as the flavor, elucidating the cooking process behind it, or creating robotic assistants that help users complete that cooking process, is of essential scientific and technological value yet technically extremely challenging.

Many fundamental AI and machine learning problems, such as fine-grained object category recognition, multi-label structured prediction, image segmentation, cross-modal analysis, retrieval, and captioning, activity understanding and recognition, natural language understanding, linguistic and visual-linguistic reference resolution, multimodal grounding, task representation, inference and planning, multimodal knowledge representation and extraction, human-robot interaction, and design of novel user interfaces & user experiences, are inherently present in the context of food AI.

This workshop is a part of the 2020 International Joint Conferences on Artificial Intelligence (IJCAI-20).

Schedule

Dates

• Abstract submission deadline: Dec. 24th, 2020

• Workshop date: 12am – 5am UTC, January 8th, 2021

Topics of Interest

The workshop is interested in work on all aspects of AI and food including but not limited to the following topics:

• Multimodal food recognition, meal to ingredient level to recipe

• Multimodal estimation of food & ingredient amounts

• Multimodal estimation of meal attributes, including flavor

• Cross-modal food and recipe retrieval

• Planning in the cooking domain for recipe and dish creation

• AI and ML in novel recipe generation (computational creativity)

• Knowledge representation for food domain knowledge: recipes, cooking procedure, ingredient interaction, flavour, nutrition, and health facts

• Knowledge extraction for food domain knowledge: ontologies, recipes

• Multimodal grounding and reference resolution

• Cooking process analysis from video and images

• Interfaces for food related AI systems, UX and UI for multimodal food logging

• Food manipulation and cooking skill learning for robots (robot learning)

Speakers

See schedules above.

Call for Extended Abstracts

We solicit 1-2 page extended abstracts of past, recently completed or on-going work with contributions to any of the topics above. Submitted papers are reviewed for suitability by the organizers. If in doubt about whether your work is in scope for the workshop, please contact us directly.

Submission: A paper submission has to be in English, in pdf format, 1-2 pages (excluding references), please format your paper using IJCAI style files.

Submission site: https://cmt3.research.microsoft.com/AIxFood2020

You will be able to upload supplementary material (e.g., videos) after submitting the paper by going to the author console in CMT and clicking "Upload Supplementary Materials."

Organizers

• Vladimir Pavlovic (Rutgers University, Samsung AI Center )

• Michael Spranger (Sony)

• Shuqiang Jiang (Chinese Academy of Sciences )

• Chong-Wah Ngo (City University of Hong Kong )

• Edward J. Delp (Purdue University )

Contact

Ricardo Guerrero: ricardo.guerrero09@alumni.imperial.ac.uk

Fangda Han: fh199@cs.rutgers.edu