Paper Link: https://arxiv.org/abs/2303.08975 or  | Github: https://github.com/vainaviv/handloom 

Abstract

This paper presents LTODO (Learned Tracing of One-Dimensional Objects) a learning-based algorithm that fits spline curves to an RGB image of one or more cables, hoses, or ropes on a surface and resolves over- vs under-crossings to disambiguate cable state for downstream tasks such as inspection, diagnostics, state-based imitation, and untangling. This work focuses on cables in semi-planar configurations, where crossings involve at most 2 segments. LTODO makes use of neural networks trained with 30,000 simulated examples and 568 real examples to autoregressively estimate splines of cables and classify crossings. Experiments find that in settings with multiple identical cables, LTODO can trace and isolate any cable of interest with 80% accuracy. In single-cable images, LTODO can trace and identify knots with 77% success. When LTODO is incorporated into a bimanual robot system, it successfully untangles 64% of cable configurations across 3 levels of difficulty and ties knots, learned from demonstrations, with 80% success. LTODO demonstrates generalization to knot types and materials (rubber, cloth) not present in the training dataset. Supplementary material, which includes an annotated dataset of 500 RGB-D images of a knotted cable along with ground-truth traces, can be found at https://sites.google.com/view/cable-tracing.

Overview of LTODO

LTODO consists of two networks. One network (pink) is trained to predict the next point in the trace given a prior context window, and the other network (blue) is trained to classify over- and under-crossings with respect to a cable segment. During inference, LTODO 1) uses the pink network to autoregressively find the most likely trace (shown through a rainbow gradient, from violet to purple, depicting the sequence in which the cable is traced) and 2) performs crossing recognition using the blue network to obtain the full state of the cable (3), where red circles indicate overcrossings and blue circles indicate undercrossings. 4) The state estimate from LTODO can be used for various downstream robotics tasks.

Tracer Generalization

Varying Textures, Colors, Thicknesses, and Lengths

Tracing in Multi-cable Settings

Knot Detection

Tier A3 results are not shown as that tier is all fake knots.

Untangling Algorithm with LTODO

We first detect the endpoints and initialize the tracer with start points. If we are not able to obtain start points, we perturb the endpoint and try again. Next, we trace. While tracing, if the cable exits the workspace, we pull the cable towards the center of the workspace. If the tracer gets confused and begins retracing a knot region, we perform a partial cage-pinch dilation that will loosen the knot, intended to make the configuration easier to trace on the next iteration. If the trace is able to successfully complete, we analyze the topology. If there are no knots, we are done. If there are knots, we perform a cage-pinch dilation and return to the first step.

Untangling Rollouts