Title: From Cooking to Eating: The role of robots in the kitchen

This presentation discusses the practical aspects of integrating robots into kitchen environments. Emphasizing the need for strong manipulation skills, it addresses how robots must perceive food and its condition, optimize recipes, and simulate human consumption processes. These functionalities are crucial for advancing kitchen automation and food service industries.

Title: Sensors and Dexterous Manipulators for Food Manipulation

Food items exhibit a wide range of material properties due to inherent variations, storage conditions, and cooking methods. It is important for cooking robots to adapt their manipulation policies to food items or cooking tools based on real-time sensory feedback. In this talk, I will introduce a suite of low-cost sensors that enable a robot to perform various cooking tasks, such as vegetable cutting and sauce dispensing. Furthermore, we will discuss the design of a soft dexterous manipulator capable of safe and effective interaction with diverse food items. Hopefully these ideas can inspire others to advance the frontier in autonomous cooking robots. 

Title: Robotic Technologies for Food Factory Automation: Vision, Tactile, and FingerVision

When we try to use robotic technologies in automating food item manipulation in industrial scenario (or everyday cooking scenario), we face a tons of difficulties. There are of course difference among factory food production and everyday cooking scenarios, many of technical challenges (and solutions for them) are common. As a startup company, we have tackled to automate processes in food factories especially with emphasizing our key technology, vision-based tactile sensing and tactile-based robot hand control. As we forecasted, tactile sensing is useful in those scenario, but those tasks are still hard even with such technologies. In this talk, I will share insights from our experiences in food process automation, discussing not only the hurdles we have overcome but also the broader implications of these advancements for the industry.

Title: Cooking robotics through the lens of chefs

Daily cooking involves many complex tasks that require using the five senses and delicately controlling one's hands. In this talk, as a researcher in digital food fabrication and as a chef with experience in a culinary school and restaurants, I will organize the tasks performed in real-world kitchens and share insights into the possibilities of cooking robotics. By including discussions with chefs and food tech companies, I will highlight the challenges faced in various kitchen settings, including food factories, chain restaurants, high-end restaurants, and homes. I hope this presentation will help robotics researchers in identifying new research topics.

Title: How close are we to humanoid household robots assisting in Cooking Tasks?

Cooking tasks present a multifaceted challenge for humanoid robots, given the versatility of required skills and the complexity of humanoid robots. Kitchens are dynamic, versatile and unpredictable environments. Cooking tasks requires versatile skills and the ability to adapt to different ingredients and tools. Cooking robots must be able to perform versatile skills, from peeling, cutting and chopping to pouring, scooping, stirring. Mixing and wipping. In addition, cooking robots must be able to reason about ingredients quantities while ensuring food safety and taste preferences. To master these tasks demands advancement in perception, manipulation, decision-making capabilities.  The talk will discuss progress in learning versatile skills for kitchen tasks, including learning tasks models for from human demonstration, learning from experience and human feedback as well as the planning and execution of complex kitchen tasks on humanoid robots.

Title: Learning Long-Horizon Manipulation Policies for Assistive Feeding

Robot-assisted feeding has the potential to improve quality of life for millions of individuals who lack the ability to feed themselves independently due to upper mobility impairments. However, realizing a truly autonomous system for feeding in-the-wild plates is incredibly difficult. The task requires a range of dexterous manipulation skills to feed different food groups, but also requires chaining these skills together over a long horizon. Finally, the robot must feed individual bites to a user in a way that is efficient, safe, and comfortable, all while being mindful of potential user preferences. In our work, we leverage multimodal perception and custom hardware to implement a library of specialized low-level skills for robotic feeding. Provided this library of skills, we demonstrate that the commonsense reasoning capabilities of vision-language models provide a powerful means to sequence available skills in a way that is efficient and informed by preferences. Finally, our systems for both food pickup, and transfer of food to the mouth, are modular enough to be composed for feeding complete plates. We look forward to applying our insights from the feeding domain to the cooking domain, where the persistent challenges of food handling and sequential planning are shared.

Title: Reverse-engineering of cooking and human taste perception using robots

Food and taste are integral to human existence, representing some of the most complex and interactive sensory-motor experiences we encounter daily. By adopting a synthetic approach to understanding these experiences, our research aims to gain additional insights into the complex interplay of low-level physiological and cognitive mechanisms, as well as uncover the nuances of higher-level phenomena, including the formation of personal preferences and the influence of culture on taste.

Historically regarded as biochemical sensing and passive perception, food tasting research has been transformed by applying robotics technologies. In our projects, we investigate human taste perception as a dynamic, proactive cognitive process, strongly involving physical and embodied interactions. This paradigm shift not only enriches our comprehension of taste but also opens up new avenues for exploring how these complex sensory-motor activities can be replicated and understood through robotic systems. This talk will introduce some of our recent case studies of cooking robots applied to better understand cooking and human taste perception.

Title: What Does It Take to Realize Cooking Robots?: From Scientific Research to Real-World Application for Cooking Robotics

In this presentation, I summarize and categorize the essential elements required for cooking robots, covering task planning, hardware, perception, and motion planning and control. I'll showcase some of my research work on task planning perception, and motion planning, illustrating advancements and challenges in enabling robots to perform culinary tasks autonomously. 

Furthermore, I'll discuss the significance of cooking robot research, addressing scientific contributions, social impact, and real-world challenges. We'll explore technical challenges, and practical issues, aiming to shed light on the journey toward kicking start the field of cooking robotics.

Title: What is Necessary to Cook Curry with a Robot?

For a robot to make curry, it needs to understand from the recipe what ingredients and steps are required, gather each ingredient, wash them, peel them, chop them, fry them in a pot, and simmer them. What kind of recognition, control, and planning are necessary for this? I will clarify what should be done now while introducing our research case.