A Hybrid Force-Position Strategy for Shape Control of Deformable Linear Objects With Graph Attention Networks

Yanzhao Yu*, Haotian Yang*, Junbo Tan , and Xueqian Wang

*Equal contribution

Tsinghua University

arXiv

GitHub

Abstract

Manipulating deformable linear objects (DLOs) such as wires and cables is crucial in various applications like electronics assembly and medical surgeries. However, it faces challenges due to DLOs' infinite degrees of freedom, complex nonlinear dynamics, and the underactuated nature of the system. To address these issues, this paper proposes a hybrid force-position strategy for DLO shape control. The framework, combining both force and position representations of DLO, integrates state trajectory planning in the force space and Model Predictive Control (MPC) in the position space. We present a dynamics model with an explicit action encoder, a property extractor and a graph processor based on Graph Attention Networks. The model is used in the MPC to enhance prediction accuracy. Results from both simulations and real-world experiments demonstrate the effectiveness of our approach in achieving efficient and stable shape control of DLOs.

Problem formulation of the considered DLO shape control task and main notations. The robot grips one end of the DLO, while the other end is fixed to a force sensor. The shape of the DLO is represented by evenly distributed keypoints. The goal is to plan the end-effector's velocity to manipulate the DLO from its current shape to the desired shape using visual data from an RGB camera and force feedback from the sensor.

Framework

Overview of the proposed hybrid force-position strategy for DLO shape control. The whole framework includes a state trajectory planner and a MPC controller. The state trajectory planner first plans the DLO's trajectory in the force space, followed by conversion to the position space. The position waypoints are then used as intermediate goals for the MPC controller, which employs a GAT-based dynamics model to predict the DLO's future state.

Dynamics Model

Schema of the EA-PE-GAT model. First, the explicit action encoder embeds the robot's actions into the graph representation of the DLO. Second, the property extractor captures the physical properties of the DLO from the historical states and output a latent graph combining the current graph and the extracted properties. Finally, the graph processor simulates the interactions between nodes in the latent graph and updates the state of the DLO.

Simulation and Real-world Experiment Setup

Setup of the simulation (based on MuJoCo) and illustration of the data collection scheme. The DLO is composed of 40 articulated capsules, with a total length of 1 m. The end-effector moves towards the destination randomly sampled in the workspace to continually generate data.