Home

Goal-oriented Communication for Fast and Robust Robotic Fault Detection and Recovery

Shutong Chen∗, Adnan Aijaz†, Yansha Deng∗

∗Department of Engineering, King’s College London, London, UK

†Bristol Research and Innovation Laboratory, Toshiba Europe Ltd., Bristol, UK

Overview

We propose a Goal-oriented Semantic Communication framework with Ligitweight Digial Twin and Small Language Model for fast Robotic Fault Detection and Recovery.

Main Contribution 1:

We define the 3D Scene Graph instead of Image/Point Cloud as the semantic information for Fault Detection, and design a semantic extractor to extract it from RGB-D images through an image processing module and a Graph Convolutional Network.

Main Contribution 2:

We fine-tune a Small Language Model to replace the Vision Language Model and Large Language Model for Fault Recovery though a LoRA fine-tuning module and a Knowledge Distillation Module.

Main Contribution 3:

We construct a Lightweight Digital Twin through object contour to enhance the robotic control precision though an Edge Point Sampling module and a Curve Fitting Module.