Food and cooking analysis present exciting research and application challenges for modern AI systems, particularly in the context of multimodal data such as images or video. A meal that appears in a food image is a product of a complex progression of cooking stages, often described in the accompanying textual recipe form. In the cooking process, individual ingredients change their physical properties, become combined with other food components, all to produce a final, yet highly variable, appearance of the meal. Recognizing food items or meals on a plate from images or videos, their physical properties such as the amount, nutritional content such as the caloric value, food attributes such as the flavor, elucidating the cooking process behind it, or creating robotic assistants that help users complete that cooking process, is of essential scientific and technological value yet technically extremely challenging.

Many fundamental AI and machine learning problems, such as fine-grained object category recognition, multi-label structured prediction, image segmentation, cross-modal analysis, retrieval, and captioning, activity understanding and recognition, natural language understanding, linguistic and visual-linguistic reference resolution, multimodal grounding, task representation, inference and planning, multimodal knowledge representation and extraction, human-robot interaction, and design of novel user interfaces & user experiences, are inherently present in the context of food AI.

This workshop is part of the 29th ACM International Conference on Multimedia (ACM MM 2021), to be held the 20th of October 2021, in Chengdu, China.

Dates

• Abstract submission deadline: 17th of August, 2021

• Accpetance notification: 24th of August, 2021

• Camera ready deadline: 31st of August, 2021

• Workshop date: 24th of October, 2021

  • 9:00 - 12:00 Tianfu Room B

Topics of Interest

The workshop is interested in work on all aspects of AI and food including but not limited to the following topics:

• Multimodal food recognition, meal to ingredient level to recipe

• Multimodal estimation of food & ingredient amounts

• Multimodal estimation of meal attributes, including flavor

• Cross-modal food and recipe retrieval

• Planning in the cooking domain for recipe and dish creation

• AI and ML in novel recipe generation (computational creativity)

• Knowledge representation for food domain knowledge: recipes, cooking procedure, ingredient interaction, flavour, nutrition, and health facts

• Knowledge extraction for food domain knowledge: ontologies, recipes

• Multimodal grounding and reference resolution

• Cooking process analysis from video and images

• Interfaces for food related AI systems, UX and UI for multimodal food logging

• Food manipulation and cooking skill learning for robots (robot learning)

Program

• (9:30 – 9:40) Opening remarks

• (9:40 – 10:00) An Integrated System for Mobile Image-Based Dietary Assessment - Zeman Shao (Purdue University), Yue Han (Purdue University), Jiangpeng He (Purdue University), Runyu Mao (Purdue University), Janine Wright (Curtin University), Deborah Kerr (Curtin University), Carol Jo Boushey (University of Hawaii Cancer Center), Fengqing Zhu (Purdue University)

• (10:00 – 10:20) 3D Mesh Reconstruction of Foods from a Single Image - Shu Naritomi (The University of Electro-Communications), Keiji Yanai (The University of Electro-Communications)

• (10:20 – 10:30) Break

• (10:30 – 11:10) Keynote - Vladimir Pavlovic

• (11:10– 11:30) A Generic Few-Shot Solution for Food Shelf-Life Prediction using Meta-Learning - Harini S (TCS Research), Jayita Dutta (TCS Research), Manasi Patwardhan (TCS Research), Parijat Deshpande (TCS Research), Shirish Karande (TCS Research), Beena Rai (TCS Research)

• (11:30 – 11:50) Analyzing and Recognizing Food in Constrained and Unconstrained Environments - Marco Buzzelli (University of Milano - Bicocca), Gianluigi Ciocca (University of Milano - Bicocca), Paolo Napoletano (University of Milano - Bicocca), Raimondo Schettini (University of Milano - Bicocca)

• (11:50 – 12:00) Closing remarks

Call for Extended Abstracts

We solicit papers with a 4 page limit (plus references) of past, recently completed or on-going work with contributions to any of the topics above. Submitted papers are reviewed for suitability by the organizers. If in doubt about whether your work is in scope for the workshop, please contact us directly.

Submission: A paper submission has to be in English, in pdf format, 4 pages (excluding references), please format your paper using ACM sigconf 2 column templates .

Submission site: https://cmt3.research.microsoft.com/ACMMM2021 (3rd Workshop on AI & Food)

You will be able to upload supplementary material (e.g., videos) after submitting the paper by going to the author console in CMT and clicking "Upload Supplementary Materials."

Organizers

• Ricardo Guerrero (Samsung AI Center - Cambridge)

• Michael Spranger (Sony)

• Shuqiang Jiang (Chinese Academy of Sciences )

• Chong-Wah Ngo (Singapore Management University)

Contact

Ricardo Guerrero: r.guerrero@samsung.com

Fangda Han: fh199@cs.rutgers.edu