Abstract

In this project, we propose a general unified tracking-servoing approach for controlling the shape of elastic deformable objects using robotic arms. Our approach works by forming a lattice around the object, binding the object to the lattice, and tracking and servoing the lattice instead of the object. This makes our approach have full 3D control over deformable objects of any general form (linear, thin-shell, volumetric). Furthermore, it decouples the runtime complexity of the approach from the objects’ geometric complexity. Our approach is based on the As-Rigid-As-Possible (ARAP) deformation model. It requires no mechanical parameter of the object to be known and can drive the object toward desired shapes through large deformations. The inputs to our approach are the point cloud of the object’s surface in its rest shape and the point cloud captured by a 3D camera in each frame. Overall, our approach is more broadly applicable than existing approaches. We validate the efficiency of our approach through numerous experiments with deformable objects of various shapes and materials (paper, rubber, plastic, foam).

Unreachable desired shape 1 (unfeasible desired shape)

Our servoing approach drives the lattice (in blue) towards a desired shape (in red). The desired shape is an unfeasible shape with different dimensions.

Unreachable desired shape 2 (limited action space) 

Our shape servoing approach drives the lattice (in blue) to a desired shape (in red) with the same dimensions; once with rotation and once without rotation.

Non-convex lattice

When the underlying object is non-convex, a uniform lattice can lead to constraints between different parts that are not directly connected. For example, in the case of a toy, links may form between the hands and legs, causing the deformation of one hand to move a leg, which is not realistic. To address this issue, the unnecessary links in the lattice should be disregarded. This is achieved by initially creating a uniform lattice around the object and then removing all tetrahedral cells and their constitutive lattice nodes that lie entirely outside the input geometry, resulting in a volumetric proxy structure. Our shape servoing approach can thus be applied to the modified lattice without any modifications.

Accuracy of Jacobian estimation over a time horizon

To determine the accuracy of our Jacobian estimation, we conducted three experiments, where two robot grippers at both ends of the object servoed the lattice towards the desired shape using our proposed controller. In each experiment, we compared the initial Jacobian estimation with the Jacobian updated at each timestep. We evaluated accuracy by comparing the predicted shapes in the next frame using the velocities applied to the two grippers and each of the two Jacobians. We calculated the cosine similarity between the lattice nodes of each predicted shape and the simulated shape in the next frame. A value closer to 1 indicates better alignment between the two shapes. A high accuracy especially for moderate deformations suggests that our Jacobian estimation could be applicable for a certain time horizon in an MPC-style controller.