Embedded Files
Abstract
Re-grasp manipulation leverages on ergonomic tools to assist humans in accomplishing diverse tasks. In certain scenarios, humans often employ external forces to effortlessly and precisely re-grasp tools like a hammer. Drawing inspiration from this, previous studies focus on developing traditional robot controller of a robot hand to achieve a desired in-grasp sliding motion of the grasped object using passive dynamic actions (e.g., gravity). However, these in-hand sliding controllers require individual design for each object, relying on finger-object contact information for in-hand re-grasp manipulation, which limits their adaptability to diverse objects. In this paper, we propose a end-to-end sliding motion controller based on imitation learning(IL) that necessitates minimal prior knowledge of object mechanics, relying solely on object position information. Experimental results demonstrate the controller’s versatility in performing in-hand sliding tasks with objects of varying friction coefficients, geometric shapes, and masses. To expedite training convergence, we utilize a data glove to collect expert data trajectories and train the policy through Generative Adversarial Imitation Learning (GAIL). We migrate our controllers to a physical system using visual position estimation. Achieving an average success rate of 86%, our approach surpasses the baseline algorithm’s success rate of 35% of Behavior Cloning (BC) and 20% of Proximal Policy Optimization (PPO) when applied to different objects.
Task Definition
In this task, the focus is on performing in-hand position adjustment. The objective is to command the object held in hand to reach a specific goal position. By leveraging the force of gravity, the object can slide and reposition itself by adjusting the angle of wrist rotation until it reaches the desired goal position.
Methods
In our proposed methodology for learning from demonstrations (LfD) for In-hand object position adjustment, we outline a three-step process. Firstly, expert demonstrations are performed and recorded using the data glove. Secondly, a robotic agent is trained within a simulated environment using the recorded demonstrations with GenerativeAdversarial Imitation learning(GAIL). Finally, the learned policy from the simulation is transferred and implemented on the real robotic system for actual in-hand manipulation tasks.
By following this methodology, we aim to develop a reliable and efficient approach for training robotic systems to perform the in-hand manipulation task.
The first step involves acquiring expert demonstrations by skilled expert using the Data Glove(VRTRIX). These demonstrations capture the precise motions and techniques employed during the manipulation task, ensuring a high-quality training dataset.
Results
This is a successful test of the In hand object position adjustment demonstration under GAIL algorithm.
This is a failed task of the In hand object position adjustment demonstration with PPO algorithm.
We conducted separate tests for the training curves of the PPO, BC and GAIL algorithms under randomized target positions. (a-c) Training curve of different algorithm (GAIL,PPO,BC). (b-d)Validation experiments of three algorithms conducted in a simulation environment. The figure displays the results from left to right, representing the BC, GAIL, and PPO algorithms. The experiment aimed to evaluate the manipulator’s ability to adjust an object in its hand to a specified target position. The target position was a cuboid identical to the one used during the training process. The experiment was repeated 50 times for each algorithm. The success rates achieved by the three algorithms were as follows: GAIL - 0.94, BC - 0.82, and PPO - 0.52.(Gp : GoalPosition, Op : Ob jectPosition)
Additionally, we evaluated the success rates of the PPO, BC and GAIL algorithms in a simulated environment with various friction coeffient, mass and shape.
Demostration of in-hand re-grasp in simulation environment
Demostration of in-hand re-grasp in real environment
Conclusion
This paper presents a novel end-to-end in-hand re-grasp manipulation controller based on Generative Adversarial Imitation Learning (GAIL). The controller is designed to operate without reliance on finger-object contact information and exhibits the ability to transfer learned policies to objects of varying weights (120, 150 and 200g), friction coefficients (0.3, 0.4 and 0.5), and shapes (cuboid and cylinder). This significantly enhances the controller’s wide applicability. An expert data collection system is implemented, which leverages data gloves to control a robotic hand in a simulation environment and collect expert trajectories. By utilizing only six expert trajectories, the GAIL controller demonstrates faster convergence and robust performance compared to traditional baseline algorithms such as Behavioral Cloning (BC) and Proximal Policy Optimization (PPO) under the same training level. A vision-based position estimated system is developed used to collect real time object position and then the learnt policy has been deployed on the real humanoid robots and the manipulation task has been successfully completed. Overall, the proposed GAIL-based end-to-end controller for in-hand re-grasp manipulation offers improved robustness and convergence compared to traditional baseline algorithms. The utilization of expert trajectories collected through the data glove-based system contributes to the enhanced performance of the controller. Future work can focus on incorporating methods for robustness validation, error detection, and recovery, as well as addressing potential safety concerns during manipulation tasks. This can involve developing techniques to handle uncertainties, sensor noise, and unexpected events that may occur in real-world scenarios
Future work
Future work will focus on the transfer of the learned policy from simulation to the real world. The policy will be tested and evaluated on the XIAOMI humanoid robot to perform in-hand object manipulation.
Page updated
Google Sites
Report abuse