Our proposed system OceanChat can decompose natural language commands into a task sequence of controlling EcoMapper in a large-scene simulation platform HoloEco. Specifically, we stablish a three-level framework of LLM-guided task and motion planner. The event-triggered replanning module is designed to withstand unpredictable error during execution.

To accomplish AUV missions specified by human commands, we develop for OceanChat a three-level framework of closed-loop LLM-guided task and motion planning. Starting from a human command, the framework operates in three levels: LLMs interpreting the command, task planners sequencing tasks, and motion planners controlling EcoMapper in the ocean environment. Despite meticulous planning, EcoMapper may still fall into unexpected consequences in the uncertain underwater environment. Therefore, it is crucial to incorporate an event-triggered replanning module to address execution error in a closed-loop fashion. If EcoMapper status deviates from the expected outcome, the task planner instructs the motion planner to execute corrective actions.

We establish an ocean simulation platform HoloEco that offers a rich environment of diverse underwater activities. Included within it are a myriad of objects such as coral reefs, gliders, warships, underwater mountains and canyons, etc. We make available a suite of underwater applications in which a real-world AUV might engage, such as surveying coral reefs or navigating through unknown canyons. Additionally, our simulator involves an AUV model EcoMapper that closely aligns with its real-world counterpart. EcoMapper is meticulously crafted with efficient controllers like moving forward and multimodal sensors like depth maps and laser range sensors.