Kanata Suzuki - LLM for Robot Learning

The starred authors(*) are co-first authors and contributed equally.

Integration of Robot Motion and LLM based on Predictive Coding Approach

Interaction using natural language is essential for enabling flexible and generalizable robot behavior in real-world environments. With the advancement of Large Language Models (LLMs), there has been increasing interest in integrating language understanding with robot motion generation. However, in many previous studies, language models and robot motion learning frameworks are treated as separate modules, resulting in limited consistency between linguistic instructions and generated actions. This separation often prevents robots from fully leveraging semantic information during motion generation. This research aims to tightly integrate language understanding and robot motion learning by constructing a shared latent space that connects sensorimotor representations and linguistic information. Specifically, the proposed method introduces sensorimotor attention mechanisms and language-based regression within shared latent variables, enabling mutual alignment between motion and language modalities [1]. By embedding both modalities into a unified representation space, the model can generate robot actions that are consistent with natural language instructions while maintaining sensitivity to sensory inputs. This approach establishes a framework for more coherent and embodied language-conditioned robot behavior, bridging the gap between LLM-based reasoning and low-level motor control.

Kanata Suzuki, Tetsuya Ogata: Sensorimotor Attention and Language-based Regressions in Shared Latent Variables for Integrating Robot Motion Learning and LLM, Proceedings of 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'24), pp.11872-11878, acceptance rate 47.5%, Abu Dhabi, UAE, October 14-18, 2024.
- DOI: https://doi.org/10.1109/IROS58592.2024.10802349

Interactively Task Planning with Uncertainty Analysis and Questioning

Interaction under uncertainty is a fundamental challenge in collaborative tasks between humans and robots. With the rapid advancement of Large Language Models (LLMs), their application to robot task planning and perception has gained significant attention, enabling more flexible and context-aware robot behaviors. However, in many previous studies, LLM-based planning is performed in a one-shot manner without considering uncertainty in perception or execution, which can lead to unreliable plans and failures in real-world environments. This research aims to enhance long-horizon robot task planning by explicitly modeling and mitigating uncertainty through interaction. Specifically, the proposed approach introduces mechanisms for both passive and active modification of LLM-generated plans [1], as well as uncertainty-aware action planning with active questioning strategies [3]. Furthermore, the framework integrates interactive object detection to refine environmental understanding based on uncertain task plans [2]. By combining uncertainty analysis, interactive querying, and perception refinement, this research establishes a robust robot planning system that can adaptively respond to ambiguous and dynamic real-world situations.

Kazuki Hori, Kanata Suzuki, Tetsuya Ogata: Enhancement of Long-Horizon Task Planning via Active and Passive Modification in Large Language Models, Scientific Reports, 15, 7113, 2025.
- DOI: https://doi.org/10.1038/s41598-025-91448-4
Kanata Suzuki, Akane Ushizaka, Kazuki Hori, Tetsuya Ogata: Interactive Object Detection by Mitigating Uncertainty of Robot Task Plans using Large Language Model, Proceedings of 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'25), pp.9598-9605, acceptance rate 46.0%, Hangzhou, China, October 19-25, 2025.
- DOI: https://doi.org/10.1109/IROS60139.2025.11246834
Kazuki Hori, Kanata Suzuki, Tetsuya Ogata: Interactively Robot Action Planning with Uncertainty Analysis and Active Questioning by Large Language Model, Proceedings of 2024 IEEE/SICE International Symposium on System Integrations (SII'24), pp.85-91, Ha Long, Viet Nam, January 8-11, 2024.
- DOI: https://doi.org/10.1109/SII58957.2024.10417267

Bidirectional Translation between Language and Motion

Interaction between language and robot actions is a key component for achieving flexible and intuitive human-robot collaboration. With the development of pre-trained language models, word embeddings have been widely used to represent semantic information. However, these embeddings are typically learned from text corpora and lack grounding in physical interactions, making it difficult for robots to directly utilize them for action generation. This research aims to bridge the gap between linguistic representations and robot actions by embedding pre-trained word representations into sensorimotor experiences. Specifically, the proposed approach enables the embodiment of pre-trained word embeddings through robot actions, allowing semantic concepts to be grounded in physical motion [1]. Furthermore, the framework extends to learning bidirectional translation between natural language descriptions and robot actions using a small amount of paired data [2]. By establishing a shared representation that connects language and motion in both directions, this research enables robots to both interpret instructions and describe their actions, even under limited supervision. This contributes to the realization of more data-efficient and semantically grounded robot learning systems.

Minori Toyoda, Kanata Suzuki, Hiroki Mori, Yoshihiko Hayashi, Tetsuya Ogata: Embodying pre-trained word embeddings through robot actions, IEEE Robotics and Automation Letters, vol.6, no.2, pp.4225-4232, 2021 (presented at ICRA'21).
- DOI: https://doi.org/10.1109/LRA.2021.3067862
Minori Toyoda*, Kanata Suzuki*, Yoshihiko Hayashi, Tetsuya Ogata: Learning Bidirectional Translation between Descriptions and Actions with Small Paired Data, IEEE Robotics and Automation Letters, vol.7, no.4, pp.10930-10937, 2022 (presented at IROS'22).
- DOI: https://doi.org/10.1109/LRA.2022.3196159