Reproducing Robot Manipulation: Developing Guidelines to Improve Accessibility and Reproducibility of Open-Source

A half-day workshop at the IEEE/RSJ International Conference on Intelligent Robots & Systems (IROS) 2026
Sunday, September 27, 2026, 1:00 - 5:30pm

Executing grasping and manipulation tasks with a robot system requires the integration of multiple software components including perception modules and task planning. While the advent of open-source contributions in robotics research should support this endeavor, there are many roadblocks to this process. Middleware like ROS has provided a common software platform for robotics research, yet issues still persist at both the core open-source code level and the ROS package level. There are a variety of open-source software components available including those for scene segmentation, pose estimation, object detection, object selection, grasp planning, motion planning, and control, as well as more advanced components that combine these functionalities like VLMs and VLAs. Regardless of the type of component, though, many open-source repositories lack materials to support reproduction tests to assess the functionality of the reproduced code, pretrained model checkpoints, ROS wrappers, and other utilities required for integration with other components.

As part of the COMPARE Ecosystem (https://robot-manipulation.org/), this workshop will bring together developers and users of open-source software/code for robot manipulation and hold in-depth discussions on recommendations and best practices to improve the accessibility, integration, modularity, and reproducibility of open-source. The content of the workshop is split into two topics: (1) Developing and Sharing Components, and (2) Integrating Components Together. For each topic, there will be 1-2 invited talks from users and developers of open-source, followed by guided breakout discussions facilitated by workshop organizers, then group reporting and review, ending with synthesizing the results into a draft set of guidelines and next steps.

The proposed workshop builds upon a series of workshops previously conducted to scope and develop the COMPARE Ecosystem (RSS 2025, RSS 2023, ICRA 2023, ROS-I 2023, HRI 2023). Prior workshops were broader in scope and considered many aspects of open-source and benchmarking, including software, hardware, datasets, object sets, and performance evaluation protocols. For IROS 2026, we narrow the focus strictly to open-source software components, aiming to bring together developers and users of open-source software/code for robot manipulation and hold in-depth discussions on recommendations and best practices to improve the accessibility and reproducibility of open-source.

Speakers

Enric Cervera

Universitat Jaume I

[invited, confirmed]

Rachel Holladay

University of Pennsylvania

[invited, confirmed]

Sebastian Castro

Respite Robotics

[invited, confirmed]

Ingo Keller

The National Robotarium

[invited, tentative]

Participation

At the start of the workshop, all participants will be asked to identify themselves according to a set of roles that best describes their interactions with open-source (e.g., user, developer, advocate) and the type of open-source software components they are most interested in (e.g., perception, planning, VLMs, etc.). During the breakout discussions, workshop participants will be split into groups of 5-10 people paired with a facilitator from the organizer team. For topic (1), we will aim to group people together with similar open-source interests towards identifying commonalities, and for topic (2) we will mix together people with varied interests given that the topic is about integration. During breakout discussions, the facilitator will guide a discussion amongst the participants to:

Identify criteria used to judge the success of reproducing and integrating open-source components,
Common issues encountered when attempting to reproduce and integrate open-source components with others in a pipeline,
Describe features of an open-source repository that would increase confidence that attempting to reproduce it would be successful,
Make recommendations on best practices – either as an existing solution or suggestions to research and develop a future one – to mitigate these issues, and
Solicit participants to conduct some of the activities discussed post-workshop, such as working to upgrade any open-source repositories they maintain to meet the guidelines developed during the workshop.