RTSP-Net: RL-Based Robotic Task Sequencing for Robotic Manipulators in Obstacle-rich Environments 

Abstract: The Robotic Task Sequencing Problem (RTSP) in the context of manipulators involves determining both the order in which a set of targets are visited and the specific configuration for reaching each target, accounting for multiple feasible poses arising from kinematic redundancy. Existing approaches frequently rely on straight-line distance approximations to estimate motion costs, an assumption that can be highly inaccurate in cluttered environments. While precomputing pairwise motion costs is practical in static industrial settings with a limited number of fixed targets, it becomes prohibitively expensive in applications such as inspection, cleaning, and disinfection, where robots often encounter previously unseen environments. To address this, we propose RTSP-Net, trained via deep hierarchical  reinforcement learning, that takes as input the environment represented as a point cloud, along with target points and their associated manipulator configurations, and outputs both the visiting sequence and the corresponding configuration for each target. RTSP-Net is trained across six distinct environments with varying obstacle layouts, using a neural cost model to provide realistic path-cost-based rewards during training. Experiments demonstrate that RTSP-Net achieves up to a 26% reduction in path length for in-distribution environments and up to a 22% reduction in out-of-distribution scenarios. Crucially, while traditional iterative baselines scale poorly, RTSP-Net maintains sub-second performance across all problem sizes, delivering up to 27x computational speedup.