Neural Network

For soft robots, neural networks are effective solutions. Neural networks have nonlinear activation functions and are good at coping with nonlinear data. Recurrent neural networks can solve time realted problems thanks to the recurrent struccture.

Recently, considerable efforts have been focused on NN applications in the soft robot field. In the early years, Extreme Learning Machines (ELM) and Radial Basis Function (RBF) were popular choices. Nowadays, researchers prefer MultiLayer Perceptron (MLP) and Recurrent Neural Network (RNN) due to their generalized and sequence-related structures, respectively. Moreover, for some special proposes like image processing, AutoEncoder (AE) and Convolutional Neural Network (CNN) are utilized.

To summarize, owing to the large variety of NN structures, they are attractive to soft robot modeling and control. MLP fulfills various tasks with the help of its generalized estimation ability, and RNN is a viable choice for sequential information processing, which is one of the main problems in this area. For most issues in soft robot control, it is highly possible to find a related NN solution. However, such models require a large amount of data due to their complicated structures, and it is infeasible to update them online.

Figure 6. Diagrams of (a) MLP, (b) RNN, and (c) CNN. MLP is composed of multiple layers. Parameters W_*, B_* in one layer are in parallel and can be trained simultaneously. RNN takes the input data x sequentially, and each bar shares the same weights W_S , W_x, W_y , b_x, and b_y . CNN obtains matrix with channel C_1, height H_1, and width W_1 as input. In the network feedforward, the channel number improves while the width and height decrease using kernels. Finally, CNN outputs a matrix with dimension (C_n, 1, 1), and a fully connected layer is employed to map to the target dimension k.

Table 4. Neural Network Paper Comparison.

Paper List:

ELM:

14JQ: Queißer J F, Neumann K, Rolf M, et al. An active compliant control mode for interaction with a pneumatic soft robot[C]//2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2014: 573-579.

ELM is applied for control considering the actuation range and directions.

16RR: Reinhart R F, Steil J J. Hybrid mechanical and data-driven modeling improves inverse kinematic control of a soft robot[J]. Procedia Technology, 2016, 26: 12-19.

Based on the robot in 14JQ, this paper compares the linear model, ELM performance on the model and error approximation.

23XW: Wang X, Dai J, Tong H S, et al. Learning-based visual-strain fusion for eye-in-hand continuum robot pose estimation and control[J]. IEEE Transactions on Robotics, 2023.

Due to its lightweight, ELM is applied for pose estimation based on optical sensor signals.

RBF:

14AM: Melingui A, Escande C, Benoudjit N, et al. Qualitative approach for forward kinematic modeling of a compact bionic handling assistant trunk[J]. IFAC Proceedings Volumes, 2014, 47(3): 9353-9358.

This paper compares RBF and MLP on robot forward modeling.

17RR: Reinhart R F, Shareef Z, Steil J J. Hybrid analytical and data-driven modeling for feed-forward robot control[J]. Sensors, 2017, 17(2): 311.

Based on 16RR, RBF is introduced in the paper.

MLP

07DB: Braganza D, Dawson D M, Walker I D, et al. A neural network controller for continuum robots[J]. IEEE transactions on robotics, 2007, 23(6): 1270-1277.

This is the first paper utilizing NN in the soft robot field. This work designs a particular parameter updating strategy for control instead of the backpropagation widely applied now.

19CC: Cheng C, Cheng J, Huang W. Design and development of a novel SMA actuated multi-DOF soft robot[J]. IEEE Access, 2019, 7: 75073-75080.

This paper sets strong constraints on each node in each layer of NN.

20PH: Hyatt P, Killpack M D. Real-time nonlinear model predictive control of robots using a graphics processing unit[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 1468-1475.

This paper applied an MLP containing residual structures like ResNet.

22GF: Fang G, Tian Y, Yang Z X, et al. Efficient jacobian-based inverse kinematics with sim-to-real transfer of soft robots by learning[J]. IEEE/ASME Transactions on Mechatronics, 2022, 27(6): 5296-5306.

This paper connects two MLPs for forward modeling and sim2real, respectively.

20AK: Kuntz A, Sethi A, Webster R J, et al. Learning the complete shape of concentric tube robots[J]. IEEE transactions on medical robotics and bionics, 2020, 2(2): 140-147.

Rotations and translations of concentric tube robots are utilized in MLP for robot shape estimation.

19RG: Grassmann R, Burgner-Kahrs J. On the Merits of Joint Space and Orientation Representations in Learning the Forward Kinematics in SE (3)[C]//Robotics: science and systems. 2019.

This paper compares different joint space forms of the concentric tubes as input on the forward modeling estimation tasks.

17HJ: Jiang H, Wang Z, Liu X, et al. A two-level approach for solving the inverse kinematics of an extensible soft arm considering viscoelastic behavior[C]//2017 IEEE international conference on robotics and automation (ICRA). IEEE, 2017: 6127-6133.

This paper employs PCC parameters for multi-segment soft robot control.

23JL: Liu J, Borja P, Della Santina C. Physics‐Informed Neural Networks to Model and Control Robots: A Theoretical and Experimental Investigation[J]. Advanced Intelligent Systems, 2023: 2300385.

MLP in this paper is used to estimate physical parameters in Lagrangian and Hamiltonian dynamics.

22FP: Piqué F, Kalidindi H T, Fruzzetti L, et al. Controlling soft robotic arms using continual learning[J]. IEEE Robotics and Automation Letters, 2022, 7(2): 5469-5476.

MLP in this paper takes data from multiple time steps as input to deal with hysteresis.

RNN:

15AM: Melingui A, Lakhal O, Daachi B, et al. Adaptive neural network control of a compact bionic handling arm[J]. IEEE/ASME Transactions on Mechatronics, 2015, 20(6): 2862-2875.

This is the first paper applying RNN, modified Elman NN, which restores information in previous steps with context nodes for dynamic control.

18TT: Thuruthel T G, Falotico E, Manti M, et al. Stable open loop control of soft robotic manipulators[J]. IEEE Robotics and Automation Letters, 2018, 3(2): 1292-1298.

NARX is leveraged in this paper for dynamic control.

21ZD: Ding Z Y, Loo J Y, Baskaran V M, et al. Predictive uncertainty estimation using deep learning for soft robot multimodal sensing[J]. IEEE Robotics and Automation Letters, 2021, 6(2): 951-957.

LSTM is used to estimate the value and variance of robot shape and external force.

22DW: Wu D, Ha X T, Zhang Y, et al. Deep-learning-based compliant motion control of a pneumatically-driven robotic catheter[J]. IEEE Robotics and Automation Letters, 2022, 7(4): 8853-8860.

The LSTM in this paper gets data from multiple time steps into each LSTM unit.

23ZCb: Chen Z, Ren X, Bernabei M, et al. A Hybrid Adaptive Controller for Soft Robot Interchangeability[J]. IEEE Robotics and Automation Letters, 2023, 9(1): 875-882.

LSTM and kinematics controller are applied in this paper to compose a hybrid controller.

Special NN:

18GS: Soter G, Conn A, Hauser H, et al. Bodily aware soft robots: integration of proprioceptive and exteroceptive sensors[C]//2018 IEEE international conference on robotics and automation (ICRA). IEEE, 2018: 2448-2453.

Autoencoder is utilized to extract features from the images of soft robots and estimate the robot's shape.

23EAb: Almanzor E, Ye F, Shi J, et al. Static shape control of soft continuum robots using deep visual inverse kinematic models[J]. IEEE Transactions on Robotics, 2023.

CNN controls robot shapes based on robot images.

23UY: Yoo U, Zhao H, Altamirano A, et al. Toward zero-shot sim-to-real transfer learning for pneumatic soft robot 3d proprioceptive sensing[C]//2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023: 544-551.

CNN is utilized to estimate robot shape based on the images of robot inner chamber.

19YZ: Zhang Y, Gao J, Yang H, et al. A novel hysteresis modelling method with improved generalization capability for pneumatic artificial muscles[J]. Smart materials and structures, 2019, 28(10): 105014.

Time sequence data are rearranged into a 2D matrix for dynamic modeling, and the spatial relationship among different elements infer the time sequence.

17YC: Chen Y, Xu W, Li Z, et al. Safety-enhanced motion planning for flexible surgical manipulator using neural dynamics[J]. IEEE Transactions on Control Systems Technology, 2016, 25(5): 1711-1723.

3D NN is applied for motion planning.

23ZCd: Chen Z, Bernabei M, Mainardi V, et al. A Novel and Accurate BiLSTM Configuration Controller for Modular Soft Robots with Module Number Adaptability[J]. arXiv preprint arXiv:2401.10997, 2024.

BiLSTM is applied for modular soft robot control to deal with the spatial sequence.

23SS: Sapai S, Loo J Y, Ding Z Y, et al. A Deep Learning Framework for Soft Robots with Synthetic Data[J]. Soft robotics, 2023, 10(6): 1224-1240.

A generative adversarial network (GAN) is utilized for synthetic data.

Page updated

Google Sites

Report abuse