Hierarchical Skilled Agents for Knot-Tying Without Human Demonstrations
Guy Freund1, Tom Jurgenson2, Matan Sudry2, and Erez Karpas2,3
1Reichman University, 2Technion – Israel Institute of Technology, 3The University of Texas at Austin
Accepted to IEEE International Conference on Robotics and Automation (ICRA) 2026
0501.mp4
The Trefoil Knot is the only prime knot with crossing number 3.
It can be obtained by joining the two loose ends of the Overhand Knot.
The video shows the Reidemeister path: R1 → Cross → Cross
0505.mp4
The Figure-8 Knot is the only prime knot with crossing number 4.
It can be obtained by joining the two loose ends of the Figure-8 Knot.
The video shows the Reidemeister path: R1 → Cross → Cross → Cross
Robotic knot-tying represents a fundamental challenge in robotics due to the complex interactions between deformable objects and strict topological constraints. We present TWISTED-RL, a framework that improves upon the previous state-of-the-art in demonstration-free knot-tying (TWISTED), which smartly decomposed a single knot-tying problem into manageable subproblems, each addressed by a specialized agent. Our approach replaces TWISTED's single-step inverse model that was learned via supervised learning with a multi-step Reinforcement Learning policy conditioned on abstract topological actions rather than goal states. This change allows more delicate topological state transitions while avoiding costly and ineffective data collection protocols, thus enabling better generalization across diverse knot configurations. Experimental results demonstrate that TWISTED-RL manages to solve previously unattainable knots of higher complexity, including commonly used knots such as the Figure-8 and the Overhand. Furthermore, the increase in success rates and drop in planning time establishes TWISTED-RL as the new state-of-the-art in robotic knot-tying without human demonstrations.
TWISTED-RL System Architecture. Top row: unmodified TWISTED components showing the reachable configurations and high-level planning. Bottom row: TWISTED-RL low-level execution with three episodes showing progression between crossing numbers. First episode: 0 → 1 with R1 action, second episode: 1 → 2 with Cross action, third episode: 2 → 3 with R1 action, each using its specialized agent. Red arrows indicate MuJoCo simulation steps.
TWISTED-RL demonstrates substantial improvements over TWISTED across all evaluation scenarios. While both methods achieve perfect performance on simple 3-Easy knots, TWISTED-RL variants solve tasks approximately 3× faster. The performance advantage becomes more pronounced on challenging knots: TWISTED-RL-C achieves 44% success on 3-Med, 46% on 3-Hard, and 61% on the diverse 3-Eval set, while TWISTED fails completely (0% success) on all harder test cases. With extended runtime, TWISTED-RL-C's performance on 3-Med increases to 62%, whereas TWISTED shows no improvement. On complex 4-crossing knots beyond the training distribution, TWISTED-RL-C achieves 26% success compared to TWISTED's 10%.
Average Success rate (%) of TWISTED vs TWISTED-RL variants across 5 different seeds.
Left: Anytime performance analysis of TWISTED and TWISTED-RL variants on the 3-Easy test set. Right: Anytime performance analysis of TWISTED-RL-C across all test sets. The X-axis is the runtime in seconds, and the Y-axis is the cumulative success rate (%).
If you find TWISTED-RL useful in your research, please consider citing our work:
@inproceedings{freund2026twistedrl,
title = {TWISTED-RL: Hierarchical Skilled Agents for Knot-Tying without Human Demonstrations},
author = {Freund, Guy and Jurgenson, Tom and Sudry, Matan and Karpas, Erez},
booktitle = {Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)},
year = {2026}
note = {To appear}
}