Results

In the Implementation Section, we have separately demonstrated the successful Peg-in-Hole manipulation using the pick-and-place target poses from Diffusion EDF and the Geometric Impedance Controller with the learned gain scheduling policy to place the peg in the hole. However, due to the fundamental issue of target poses generated by Diffusion EDF being insufficiently accurate for the Geometric Impedance Controller with the learned gain scheduling policy to effectively place the peg, this section presents a pipeline where the pick-and-place operation works in an open-loop configuration.

Challenges with Point Clouds and Diffusion EDF

Diffusion EDF is highly sensitive to the quality of input point clouds. In our setup, the scene point clouds were generated by merging data from two fixed cameras, resulting in sparse point clouds compared to the dense SLAM-based point clouds used in the original paper. These point clouds also suffered from noisiness, shadowing points, and unwrapping points due to the inherent limitations of time-of-flight cameras. We tried to minimize camera extrinsics errors, but we could not eliminate them entirely and had to proceed after achieving a certain level of accuracy. The sparse and noisy point clouds used to obtain pick and place poses from Diffusion EDF likely introduced significant errors in the target poses.

Propagation of Errors in Open-Loop Configuration

Since our system operates in an open-loop configuration, the errors in target poses were directly propagated to the Geometric Impedance Controller (GIC). The GIC has a precision threshold of 5mm and is incapable of mitigating larger target pose errors. This limitation impacted the system's ability to utilize the target place poses acquired from Diffusion EDF as reference points for GIC to insert the peg.

Robot-Side Limitations and Gain Scheduling Issues

On the robot side, additional challenges arose due to the custom controller mode's behavior. The controller only updated the gains when the arm was stationary, which created issues when high gain commands were given after the robot reached a stationary state. In these cases, the robot would suddenly move at high speed, making it difficult to ensure smooth and controlled movements. This behavior complicated the data collection process, as gain tuning had to be done manually based on visual feedback of the robot’s movements. This limitation also affected running the learned gain scheduling policy, as the robot could not keep up with the rate of gain updates, making it unable to respond properly to the interaction with the environment.

These challenges highlight the need for improvements in both the perception pipeline and the robot control system to enhance the accuracy of the manipulation task. Our next step is to solve this fundamental issue and explore different feedback methods to close the loop, enabling Diffusion EDF and the Geometric Impedance Controller with the learned gain scheduling policy to work together in a closed-loop system.

Our project integrates these components into a complete open loop pipeline:

Collect scene point cloud
Infer and heuristically choose the most upright and feasible target pick place from Diffusion EDF
Move the to the target pick place and pick the peg
Move in front of a camera to obtain 4 grasp point clouds and stitch them to one combined point cloud
Infer target place pose from Diffusion EDF
Move to target place pose
Use Spiral Search to mitigate the target place pose error, combined with the Geometric Impedance Controller to search for the hole and place the peg

Below is the video of our full pipeline.

full_edited_X2.mp4

Finally, we are able to use Diffusion EDF to reliably pick up a peg, come near the hole, and use spiral search along with Geometric Impedance Controller to insert it into a 1mm tolerance hole.

Page updated

Report abuse