Prof. Yoko Yamakata received her Ph.D. in Informatics from Kyoto University in 2006. She has held academic positions at Kyoto University and the University of Tokyo and has been a Professor since March 2024. Her research focuses on multimedia processing, particularly image recognition and natural language processing, with pioneering contributions to food computing since 2002. Her outreach includes the development of “FoodLog Athl,” a food image recognition app with over 120,000 meal records, and a semi-automated nutrition calculator using depth and segmentation models. She has also explored environmental metrics such as Total Material Requirement (TMR) in food. Her tools for semantic recipe analysis support multilingual research and have enabled innovations in recipe retrieval and generation. She has actively contributed to the multimedia community through leadership roles in major conferences, including MMM, ACM Multimedia Asia, ICMR, MIPR, and CEA.

Food Management Application Based on Image Recognition Technology

  She has developed the food management application "FoodLog Athl," which is based on food image recognition in collaboration with Prof. Kiyoharu Aizawa of the University of Tokyo [1]. This application has been reviewed by the university's ethics committee and is available to the public through the App Store and Google Play, with over 120,000 meal image records already registered. In 2019, in preparation for the Tokyo Olympics, FoodLog was introduced to the buffet at the National Institute of Sports Science, demonstrating that accurate nutritional values could be calculated through food image recognition [2].

[1] Nakamoto, et al. (5th of 6 authors), “FoodLog Athl: Multimedia Food Recording Platform for Dietary Guidance and Food Monitoring,” In Proc. of MMAsia '22, No. 43, https://doi.org/10.1145/3551626.3564978, 2022.

[2] M. Anzawa, et al. (3rd of 6 authors), “Recognition of Multiple Food Items in a Single Photo for Use in a Buffet Style Restaurant,” IEICE Trans. Information and Systems, E102-D, 410-414, 2019.

Development of a Semi-Automated Nutrition Calculation App Using a Food Ingredient Estimation Model

 To create nutritional records in the same way that dietitians assess patients' meals in medical and caregiving settings, she is developing a model that estimates a list of food ingredients from meal images. By inputting both color images and depth images of meals, the model estimates energy (kcal), weight, protein, carbohydrates, and fats. By using a segmentation model to focus on the meal area, the model has achieved the world's highest performance in this task [3]. Additionally, she has developed a model that generates a food ingredient list in the same format used by dietitians for nutritional assessment, and it is scheduled to be presented in the Demo Track at ACM Multimedia, a top conference in the multimedia field [4].

[3] S. Parinayok, Y. Yamakata, K. Aizawa, “Open-Vocabulary Segmentation Approach for Transformer-Based Food Nutrient Estimation,” MMAsia '23, Article 78, pp. 1–7, 2023.

[4] Y. Yamakata, et. al, “Measure and Improve Your Food: Ingredient Estimation Based Nutrition Calculator,” ACMMM’24 Technical Demo, 2024 (to appear).

Semantic Analysis of Recipe Texts Using Meal Image Recognition and Natural Language Processing Technologies

The principal investigator began pioneering AI research in the field of food in 2002, providing a corpus and tools for semantic analysis of recipes. These tools support not only Japanese but also English and Chinese and have been utilized in recipe research worldwide. Additionally, the researcher has developed methods for recipe retrieval through graph representation, recipe synthesis, and generating recipe texts from a series of cooking photos [5].

[5] Nishimura et al. (5th of 6 authors), “Structure-Aware Procedural Text Generation from an Image Sequence”, IEEE Access, 9, 2125-2141, 2021.