Robotic systems have advanced significantly, yet effective planning in unstructured environments remains a substantial challenge. This paper introduces ConceptBot, a novel planning system that integrates Large Language Models (LLMs) with Knowledge Graphs (KGs), particularly ConceptNet, to enhance task decomposition and context understanding. Unlike traditional methods reliant on pre-programmed models or extensive training on specialized datasets, ConceptBot features a modular architecture with three components: Object Properties Extraction (OPE), User Request Processing (URP), and Planner. OPE identifies and extracts object properties relevant to the task, leveraging KGs to provide contextual relationships for improved scene understanding. URP interprets natural language instructions, using KGs to better understand the user's needs and determine how to satisfy them effectively. The Planner synthesizes this information to generate executable pick-and-place policies that are contextually appropriate and feasible.
To evaluate its effectiveness, ConceptBot was compared with Google's SayCan, demonstrating superior performance in handling ambiguous instructions and generating feasible policies. Experiments were conducted in a simulative environment using PyBullet, ViLD, and CLIPort, as well as in the IDSIA laboratory with the Franka Emika Panda robotic arm, Intel RealSense camera, and YOLO-based object detection. Results highlight ConceptBot’s exceptional adaptability, accurate user request interpretation, and robust handling of ambiguous objects, achieving high performance in pick-and-place operations without additional specialized training.