Robust Learning of Tactile Force Estimation through Robot Interaction
Balakumar Sundaralingam, Alexander (Sasha) Lambert, Ankur Handa, Byron Boots, Tucker Hermans, Stan Birchfield, Nathan Ratliff, Dieter Fox
NVIDIA, University of Utah, Georgia Institute of Technology and University of Washington
Abstract
Current methods for estimating force from tactile sensor signals are either inaccurate analytic models or task-specific learned models. In this paper, we explore learning a robust model that maps tactile sensor signals to force. We specifically explore learning a mapping for the SynTouch BioTac sensor via neural networks. We propose a voxelized input feature layer for spatial signals and leverage information about the sensor surface to regularize the loss function. To learn a robust tactile force model that transfers across tasks, we generate ground truth data from three different sources: (1) the BioTac rigidly mounted to a force torque~(FT) sensor, (2) a robot interacting with a ball rigidly attached to the same FT sensor, and (3) through force inference on a planar pushing task by formalizing the mechanics as a system of particles and optimizing over the object motion. A total of 140k samples were collected from the three sources. We achieve a median angular accuracy of 3.5 degrees in predicting force direction~(66% improvement over the current state of the art) and a median magnitude accuracy of 0.06~N~(93% improvement) on a test dataset. Additionally, we evaluate the learned force model in a force feedback grasp controller performing object lifting and gentle placement.
[April 2020] Updated Force Estimation Model with Confidence Output
Using multi-sample dropout, we were able to get better accuracy and also get a confidence measure on the estimated force, more details in this preprint: arxiv
The below figure shows the improvement using multi-sample dropout ('Dropout') over our ICRA 2019 model:
Dataset Collection Protocol
We collected ground truth data from three sources as described in the paper. We provide additional details on the data collection here.
For the planar-pushing data collection, we tracked the pose of the BioTac sensor and the pushed box by using DART[1] with contact prior to prevent the BioTac mesh from penetrating the box due to noisy image from the depth camera. We used rosbag to bag all the required topics during our data collection process. DART tracks at 60Hz while the biotac sensor readings are at 100Hz. We smooth and resample the pose data by applying a third order Savitzky–Golay filter with a window size of 51. We compute the velocity and acceleration of the box and the biotac from the smoothed and resampled pose data.
Due to time synchronization issues in ROS, at every time-step we additionally check if the biotac is not in contact with the box and project the biotac to be in contact by moving the pose along the smoothed spline function. This projection ensures the force inference optimization computes a realistic force, given the instantaneous object motion. We compute the force only when the object is in motion. We assume the object is static when the L-1 norm of instantaneous object velocity is less than 5e-3.
Force Feedback for Object Manipulation
In the paper we only reported the results when the force magnitude is set to 2N. When we tested a force magnitude of 0.75N, only our method was able to successfully lift the objects. The other methods failed to lift, hence we couldn't perform placement experiments at 0.75N for the other methods. At 0.75N, our method obtained similar results to the reported 2N.
Error Analysis Plots:
In the paper, we only reported scaled errors~(Fig.5 in the submitted paper) to analyze the improvement of our proposed approach on learning the force model.
We show additional plots here. Specifically:
- We plot the absolute magnitude error without scaling, to show the accuracy of our proposed model.
- We also report the 2d projected error for the planar-pushing source as we only have ground truth from planar motion.
No Projection for planar-pushing
We see that with a model trained on just rigid-ft, the magnitude error is very large for the learned models. The analytic model has lower error in the absolute magnitude across different data sources as the linear model is able to capture the magnitude. The analytic model performs worse in the direction.
2d projected error for planar-pushing
By computing the 2d projected error for the planar pushing source, we see the error computed is smaller than when no projection is used.
We show a zoomed version of the magnitude error without the rigid-ft for clarity.
No projection for planar-pushing
2d projected error for planar-pushing
Effect of network:
No projection for planar-pushing
