Research Projects

dVRK-based teleoperation of a CTR robot with stereovision feedback for neurosurgery

We propose a robotic system with a flexible end-effector and stereovision feedback for neurosurgery by integrating our previously-developed Concentric Tube Robot (CTR), the da Vinci Robot Research Kit (dVRK) and a customized dual endoscope camera subsystem. The CTR manipulator was teleoperated with the dVRK master tool manipulator (MTM). A virtual motion boundary was applied for the MTM by haptic feedback based on the CTR’s workspace to guide the operator to control the CTR within its motion range. The manipulation performance of the proposed system was experimentally evaluated and the results of that showed under the stereovision feedback the manipulation accuracy of the CTR is 2.8 mm and the image transmission latency is 1.5 seconds. This preliminary study suggests that our proposed system has the potential of improving surgeons’ manipulation performance in robot-assisted minimally invasive neurosurgery.

Related Publications: 

Changyan He, Robert H. Nguyen, Eric Diller, James Drake, and Thomas Looi, “dVRK-based teleoperation of a CTR robot with stereovision feedback for neurosurgery”, Hamlyn Symposium 2023 (To appear)

A Magnetic Continuum Robot with Independently Actuated Multi-Magnets

Continuum robots are gradually being considered for a variety of medical applications since their narrow curvilinear shape makes them well suited to pass through body lumens, natural orifices, or small surgical incisions for the performance of minimally invasive procedures. Existing research on continuum robots mainly focus on the guidance of the robot tip. Active shape control, however, is yet to be investigated, which is important as it can enable the robot to dynamically adjust its shape and steer around obstacles.

Herein, a new type of continuum robot, referred to here as multi-magnet continuum robot (MMCR) is proposed for minimally invasive procedures. MMCR is composed of a set of two telescoping straight Nitinol tubes with two cylindrical permanent magnets that are attached at each tip of the two tubes separately. The two magnets are actuated remotely by a magnetic field that is generated by an electromagnetic navigation system,  and serve as two active nodes to change the MMCR’s shape. 

Related Publications: 

C. He, et al., “A Magnetic Continuum Robot with Independently Actuated Multi-Magnets”, (In Preparation)

A Hybrid Steerable Robot with Magnetic Wrist for Minimally Invasive Epilepsy Surgery

Dexterity is demanded for an endoscopic tool to handle complicated procedures in neurosurgery, e.g., removing diseased tissue from inside the deep brain along a tortuous path. Current robotic tools are either rigid or lack wristed motion ability at the tip, leading to limited usage in minimally invasive procedures. 

Therefore, a hybrid steerable robot with a magnetic wristed forceps is proposed to provide enhanced dexterity for endoscopic epilepsy surgery. A set of three precurved Nitinol tubes with concentric deployment, called a concentric tube robot (CTR), serves as a 6 degrees-of-freedom (DoF) robotic positioner. The magnetic wristed forceps is composed of a rotational wrist joint, and forceps at the tip, both of which are actuated remotely by magnetic fields. The magnetic wrist and forceps provide an extra rotational DoF and a gripping DoF on top of the CTR, respectively. The magnetic wrist and gripper are designed to have a hollow channel along their common axis, inside which a soft tube is deployed as a second functional tool for irrigation or suction. An electromagnetic navigation system (eMNS) with 8 coils is used to create the quasi-static magnetic fields.

Related Publications: 

Changyan He, R. Nguyen, C. Forbrigger, J. Drake, T.  Looi, E. Diller, “A Hybrid Steerable Robot with Magnetic Wrist for Minimally Invasive Epilepsy Surgery”,  IEEE International Conference on Robotics and Automation (ICRA) 2023.

Active Interventional Control Framework for Robot-Assisted Retinal Surgery

Robotics-assisted retinal microsurgery provides several benefits including improvement of manipulation precision. The assistance provided to the surgeons by current robotic frameworks is, however, a “passive” support, e.g., by damping hand tremors. Intelligent assistance and active guidance are, however, lacking in the existing robotic frameworks. 

As such, we developed an active interventional control framework to increase operation safety by actively intervening in the operation to avoid exertion of excessive forces to the sclera. The key of the control framework is that a recurrent neural network is trained to predict the occurrence of undesired events based on a short history of time series of sensor measurements, and based on the network's prediction, a variable admittance controller is activated to command the robot away from the undesired instances.

Related Publications: 

Changyan He, N. Petal, M. Shahbazi, Y. Yang, P. Gehlbach, M. Kobilarov, I. Iordachita, “Toward Safe Retinal Microsurgery: Development and Evaluation of an RNN-Based Active Interventional Control Framework”, IEEE Transactions on Biomedical Engineering, 202004, 4(67): 966-977 (FEATURE ARTICLE)

Changyan He, N. Petal, A. Ebrahimi, M. Kobilarov, I. Iordachita, "Preliminary Study of An RNN-Based Active Interventional Robotic System (AIRS) in Retinal Microsurgery." International Journal of Computer Assisted Radiology And Surgery, 201906,14(6): 945-954

Changyan He, N. Petal, I. Iordachita, M. Kobilarov, “Enabling Technology for Safe Robot-Assisted Retinal Surgery: Early Warning for Unsafe Scleral Force”, IEEE International Conference on Robotics and Automation (ICRA), 201905: 3889-3894

Automatic Light Pipe Actuating System for Bimanual Robot-Assisted Retinal Surgery

Retinal surgery is a bimanual operation in which surgeons operate with an instrument in their dominant hand and simultaneously hold a light pipe with their non-dominant hand to provide illumination inside the eye. Manually holding and adjusting the light pipe places an additional burden on the surgeon and increases the overall complexity of the procedure. 

To overcome these challenges, a robot-assisted automatic light pipe actuating system is proposed. A customized light pipe with force-sensing capability is mounted at the end effector of a follower robot and is actuated through a hybrid force-velocity controller to automatically illuminate the target area on the retinal surface by pivoting about the scleral port (incision on the sclera).

Related Publications: 

Changyan He, E. Yang, A. Ebrahimi, N. Petal, M. Shahbazi, Peter Gehlbach, I. Iordachita, “Automatic Light Pipe Actuating System for Bimanual Robot-Assisted Retinal Surgery”, IEEE Transaction on Mechatronics,  202012, 25(6): 2846-2857

Dual Force Constraint Control for Bimanual Robot-Assisted Cannulation

Retinal vein cannulation is a promising approach for treating retinal vein occlusion that involves injecting medicine into the occluded vessel to dissolve the clot. The approach remains largely unexploited clinically due to surgeon limitations in detecting interaction forces between surgical tools and retinal tissue. 

As such, a dual force constraint controller for robot-assisted retinal surgery was presented to keep the tool-to-vessel forces and tool-to-sclera forces below prescribed thresholds. A cannulation tool and forceps with dual force-sensing capability were developed and used to measure force information fed into the robot controller, which was implemented on existing Steady Hand Eye Robot platforms. The robotic system facilitates retinal vein cannulation by allowing a user to grasp the target vessel with the forceps and then enter the vessel with the cannula.

Related Publications: 

Changyan He,  A. Ebrahimi, E. Yang, M. Urias, Y. Yang, P. Gehlbach, I. Iordachita, “Towards Bimanual Vein Cannulation: Preliminary Study of a Bimanual Robotic System With a Dual Force Constraint Controller”, IEEE International Conference on Robotics and Automation (ICRA), 202005:4441-4447.

FBG Micro-Sensor Integrated Force-Sensing Instrument Development

Retinal surgery remains one of the most challenging tasks in microsurgery. The intraoperative tool-to-tissue interactive forces can be several millinewtons and well below the human hand's perception capability, which may put the fragile tissues (e.g. retinal vein) under iatrogenic injury.

To address the above challenge, we developed a sensorized cannulation tool capable of detecting both tool-to-vein puncture forces and tool-to-sclera contact forces. By combining two materials, nitinol alloy for the tool tip and stainless steel for the tool shaft, to achieve dual stiffness, the tool possesses a flexible tip to capture small vein puncture forces and a stiffer shaft to maintain straightness during use. Three segments of fiber Bragg grating sensors are calibrated to measure the transverse forces at both the tool tip and sclerotomy, as well as to determine the tool insertion depth within the eye. The root mean square error of the measurements for the force at the tip, the force at the sclerotomy, and the tool position are 0.70 mN, 1.59 mN, and 0.69 mm, respectively.

Related Publications: 

Changyan He, E. Yang, I. Iordachita, “Dual-Stiffness Force-Sensing Cannulation Tool for Retinal Microsurgery”, International Engineering in Medicine and Biology Conference (EMBC), IEEE, 201907: 3212-3216.

Evaluation of Surgeons' Performance in Robot-Assisted Retinal Surgery

The introduction of robotic assistance has the potential to enhance and expand a surgeon’s manipulation capabilities during retinal surgery, i.e., improve precision, cancel physiological hand tremors, and provide sensing information. However, surgeon performance may also be negatively impacted by robotic assistance due to robot structural stiffness and non-intuitive controls. In complying with robotic constraints, the surgeon loses the dexterity of the human hand. 

To investigate the surgeons' performance in robot-assisted operations, we conducted experimental research. In the experiments, user behavior is characterized by measuring the forces applied by the user to the sclera, the tool insertion/retraction speed, the tool insertion depth relative to the scleral entry point, and the duration of surgery. The results show that robot assistance prolongs the duration of the surgery and increases the manipulation forces applied to sclera. The collected data could be used as the force control criteria in robot-assisted retinal surgery.

Related Publications: 

Changyan He, M. Roizenblatt, N. Petal, A. Ebrahimi, Y. Yang, P. Gehlbach, I. Iordachita, "Towards Bimanual Robot-Assisted Retinal Surgery: Tool-to-Sclera Force Evaluation." In 2018 IEEE SENSORS, 2018:1701—1704.

Changyan He, A. Ebrahimi, M. Roizenblatt, N. Petal, Y. Yang, P. Gehlbach, I. Iordachita, "User Behavior Evaluation in Robot-Assisted Retinal Surgery." In 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 201808: 174-179

Changyan He, M. Roizenbllat, N. Petal, A. Ebrahimi, P. Gehlbach, I. Iordachita, “Robotic assistance affects manipulation skills in bimanual retinal surgery simulation: a tool-to-sclera force study”, Investigative Ophthalmology & Visual Science, 2019

Development of a master-slave robotic system for retinal surgery

Retinal surgery continues to be one of the most technically demanding surgeries for its high manipulation accuracy requirement, small and constrained workspace, and delicate retinal tissue. Robotic systems have the potential to enhance and expand the capabilities of surgeons during retinal surgery. As such, we developed a master-slave robot system for retinal surgery. This robotic system is designed based on characteristics of retinal cannluation surgery and analysis of the surgical workspace in eyeball. A new end-effector of two degrees of freedom (DoF) is designed and a novel remote center of motion mechanism with four DoF is adopted in the robot structure. The experiment on porcine eyes is conducted, verifying the feasibility of this system.

Related Publications: 

Changyan He, L. Huang, Y Yang, Q. Liang, Y. Li, “Research and Realization of a Master-Slave Robotic System for Retinal Vascular Bypass Surgery”, Chinese Journal of Mechanical Engineering, 201812, 31(1): 78. (Cover Paper)