RESEARCH STATEMENTS

1. Prosthetic hand control based on bio-signal

I conducted a study to control a prosthetic hand by estimating the user's intentions based on bio-signal processing and AI. Our prosthetic hand system comprises robotic hardware, electromyography (EMG) signal acquisition circuit, and a portable electronic board including motor drivers and a microprocessor for controller and data processing. I designed and manufactured each element constituting the prosthesis and built a mobile integrated system through miniaturization. I estimated the user's finger force intent based on the EMG signal and used it as a control input for the robotic hand. We evaluated the performance of the prosthetic hand through amputation patient clinical testing. We have found that users have intuitive, real-time control of their prosthetic hands. Please see the details of the study below.

1 - A: Finger force Intention estimation using EMG

To boost the usability of a robotic prosthetic hand, providing degrees of freedom to every single finger is inevitable. Under the name of simultaneous proportional control (SPC), many studies have proposed methods to achieve this goal. In this research, we propose a method to generate a regression model of a neuromuscular system called the Constrained AutoEncoder Network (CAEN) that estimates finger forces using a surface electromyogram (sEMG). Modifying the autoencoder from deep learning, the model is generated in a semi-unsupervised manner where only sEMG data and finger labels are used. In the learning process, the finger labels are used at the central layer of the network, where the three-finger forces are estimated, to prevent penetration of other finger signals to each finger node, and the network is trained in a constrained manner. This process results in independence among estimated finger forces such that the manipulability of multiple fingers is highly improved.

Related papers:

• Y. Cho, P. Kim, and K. S. Kim, "Estimating Simultaneous and Proportional Finger Force Intention Based on sEMG Using a Constrained Autoencoder", IEEE Access, July, 2020

• M. Cho, Y. Cho, and K. S. Kim, “Training Strategy and sEMG Sensor Positioning for Finger Force Estimation at Various Elbow Angles”, International Journal of Control, Automation and Systems, 20, 1621-1631, 2022

• Y. Cho, P. Kim, and K. S. Kim, “Simultaneous and Proportional Wrist Force Intent Estimation Method using Constrained Autoencoder”, 2021 International Conference on Control, Automation and Systems (ICCAS), 2021

1 - B: Wearable upper-limb prosthetics

Amputees suffer from the weight, insufficient power, and uncomfortable control methods of their prostheses. Recent studies have introduced many ideas for both hardware and software to tackle these problems. We introduce a preliminary platform of a robotic prosthetic hand system called the MSC hand that integrates effective mechanical mechanisms and intuitive control methods. The hand adopts mode-switchable twisted string actuators to provide a wide range of grasping speed (closing speed of 0.8 s) and grasping force (pinch force of 45 N) with a lightweight of 390 g. All the fingers and the thumb flex and extend actively, and the thumb is also able to abduct and adduct passively. The active fingers are controlled by surface electromyographic signals, and a learning-based neurophysiologic model is used to estimate human intention for each finger. The model provides independent intentions for each finger, so the simultaneous and proportional control of multiple fingers is possible in real-time. The performance of the MSC hand was verified through standardized experiments such as online simulation and the box and block test. In addition, a demonstration of gripping various objects was performed. The results showed rapid and precise gripping and intuitive control over the tasks.

Related papers:

• Y. Cho, Y. Lee, P. Kim, S. Jeong, and K. S. Kim, “The MSC Prosthetic Hand: Rapid, Powerful, and Intuitive”, IEEE Robotics and Automation Letters, 2021 (RA-L and ICRA presentation option)

Related research projects:

• [Project leader] Clinical Study Based High-Performance Prosthetic Hand Development and Empirical Studies (*Basic Science Research Program, Ministry of Science and ICT, South Korea)

• Development of Ultra‐light(50g), High Performance (250N, 100mm/s) Tendon Driven Robot Joint Actuation Module Based on Active‐variable‐speed Twisted‐string Actuation Mechanism and Precise Position/Stiffness Control Technology (*Ministry of Trade, Industry & Energy, South Korea)

1 - C: Bio-signal acquisition (EMG, EIM, EEG)

In addition to electromyography (EMG), we conducted research on various biosignals such as electrical impedance myography (EIM) and electroencephalogram (EEG). We designed and manufactured a signal processing board for robust signal acquisition and used it in various recognition fields. By designing a digital filter suitable for the target of each electrical signal and removing artifacts, the signal to noise ratio (SNR) was increased and it was made to be robust against external electrical stimulation. For the production of portable equipment, digital signal processing was made possible by developing a small board and using a micro controller unit (MCU). Additionally, we designed an integrated system of sensors, prosthetics, and sensory feedback through robot operation system (ROS)-based communication and controller embedding.

Related papers:

• Y. Cho, P. Kim, and K. S. Kim, “Electrical impedance myography (EIM) For multi-class prosthetic robot hand control”, 2020 International Conference on Control, Automation and Systems (ICCAS), 2020

• Y. Cho, M. Cho, Y. Lee, and K. S. Kim, “Development of wearable dry sEMG sensor and selection of optimal location through anatomical consideration for the prosthetic hand”, 2019 International Conference on Control, Automation and Systems (ICCAS), 2019

Related research projects:

• Brain Signal Based Movement Pattern Command Code Generation (*Korean Agency for Defense Development and Defense Acquisition Program Administration.)

1 - D: Sensory feedback (Electro-tactile)

We developed electro-tactile sensory feedback for sensory feedback of the prosthetic finger. By introducing the concepts of 'intermittent stimulation' and 'offset pulse' to multi-channel electro-tactile stimulation, information on the position and force of each finger was delivered to the user. We designed and fabricated a small electrical stimulation board with five channels and applied it to the prosthesis. The sensory feedback helps the user to adapt to the prosthesis more quickly and has the effect of improving the prosthesis control performance.

Related research projects:

• [Project leader] Clinical  Study  Based  High-Performance  Prosthetic  Hand  Development  and  Empirical Studies (*Basic Science Research Program, Ministry of Science and ICT, South Korea)

• [Project leader] Development of a high‐performance prosthetic robot hand based on a new twisted string actuation and its control using a neuro‐feedback human interface (*Basic Science Research Program, Ministry of Science and ICT, South Korea)

In using real-time myoelectric-controlled prostheses, the low accuracy of the user’s intention estimation for simultaneous and proportional control (SPC) makes application to real-world scenarios difficult. To overcome this barrier, I proposed a method to estimate muscle unit activation in real-time through neurophysiological modeling of the forearm. I estimated the activation of 218 muscle units in the muscle cross-section from the surface electromyogram (sEMG) acquired with six electrodes. The relationship between muscle unit action potential (MUAP), motor unit discharge timing, and measured sEMG was mathematically modeled in the form of convolution, and real-time calculation was made possible through matrix definition and simplification. Unlike the existing method using the amplitude of the sEMG signal, the proposed method can acquire more information through dimensional expansion and enables real-time visualization of muscle cross-section.

Related papers:

• Y. Cho, and P. Kim, “Real-Time Finger Force Estimation Robust to a Perturbation of Electrode Placement for Prosthetic Hand Control”, IEEE Transactions on Neural Systems & Rehabilitation Engineering (TNSRE), 2022

By proposing a mathematical model for estimating muscle unit activation and using it for intention estimation, the finger force intention can be proportionally and simultaneously estimated with high accuracy. Real-time calculations are also possible. The proposed model outperformed rather than implementing the conventional black box-type intention estimation models in terms of accuracy and independency between fingers. We conducted a target reaching experiment (TRE) to indirectly verify the real-time prosthetic control performance. The estimated finger force intention is normalized based on 50% maximum voluntary contraction (MVC) and expressed as the bar’s position in three windows for each finger. The bars can move between 0 (rest) and 1 (50% MVC). Then, we marked the target in the form of a black box with a width of 0.2 [a.u.] in the three windows. The subjects should try to keep all bars within the target simultaneously. The proposed model outperformed all other models with statistically significant differences (p<0.001).

Changes in electrode position inevitably occur depending on the time of use and removal of a prosthesis. In the case of the regression method of previous intention estimation models, acquiring new training data and retraining the model is inevitable, which requires time and effort from the user. To overcome this issue, the robustness of the model with respect to electrode position was ensured through an algorithm that compensates for electrode position change, which is one of the parameters in the proposed model for estimating muscle unit activation. it attained high performance in electrode shift compensation, and at this time, the amount of data required and the number of models utilized were small. As a result, we were able to secure the robustness of the model for estimating finger force.