RoDEO

Robot-induced Deformation in Endoscopic Operations

— Predicting Physical Tissue Responses Beyond the Field of View

About Us

Welcome to the RoDEO (Robot-induced Deformation Prediction in Endoscopic Operations ) challenge.

This challenge is a part of the IEEE RAS/EMBS 11th International Conference on Biomedical Robotics and Biomechatronics, BioRob 2026, which will be held from August 1st to 4th, 2026, Edmonton, Alberta, Canada.

Join the Challenge!

Introduction

Achieving high levels of surgical autonomy requires robust safety mechanisms to monitor and predict robotic interactions with soft tissues. In minimally invasive surgery (MIS), endoscopes provide a constrained field of view that often excludes significant portions of the operative field. During robotic manipulation, tissue deformations frequently propagate into these occluded regions, creating risks of undetected injuries such as excessive stretching or tearing. Accurately estimating the state of tissues outside the field of view (FoV) is essential for collision avoidance and stress monitoring.

This challenge invites participants to develop methods for predicting the physical state of hidden environments and organs using the DRENDS dataset. The dataset features a unique configuration where the initial frame provides a comprehensive, un-cropped view of the scene, while subsequent frames are restricted to local, cropped observations. Participants must integrate this initial global context with synchronized robotic kinematics to estimate scene geometry and organ deformation at specific temporal intervals.

Challenge Tasks in Detail

The challenge consists of two tasks. You can participate in either or both. Predictions are compared against the ground truth data from the cropped data.

Task 1: Out-of-FoV Scene Depth and Flow Extrapolation

This task focuses on the overall surgical environment. The goal is to predict the depth and motion of the background tissues that have moved out of the camera's view.

Goal: Predict the depth and 2D deformation field of the background scene outside the current FoV.
Input Data:
1. First Frame Reference: The un-cropped RGB image and full depth map from frame 1.
2. Local Observations: Cropped RGB images and local depth maps from the current sequence.
3. Robotic Kinematics: Time-series data of the robot's joint angles and tool positions.
4. Intrinsics: Camera calibration parameters.
Target Output:
1. Out-of-FoV Depth Map: The depth values for the hidden pixels at a specific frame, e.g., frame 10, 20, or 50.
2. Out-of-FoV Deformation Flow: The 2D movement (u, v) of hidden pixels from frame 1 to a specific frame, e.g., frame 10, 20, or 50.
Metrics:
1. Out-of-FoV Depth RMSE: Measures the average depth error in hidden areas.
2. Out-of-Fov Deformation AEPE: Measures the average error of the predicted 2D movement vectors.

Task 2: Out-of-FoV 3D Deformation of Manipulated Organ

This task focuses on the specific organ being handled by the robot. The goal is to recover the 3D shape of an organ as it is pulled or pushed out of the camera's view.

Goal Use a 3D model of the organ to estimate its shape in hidden areas during manipulation.
Input Data:
1. First Frame Reference: The un-cropped RGB image and full depth map from frame 1.
2. 3D Template: A static 3D mesh model of the specific organ.
3. Local Observations: Cropped RGB images and local depth maps from the current sequence.
4. Robotic Kinematics: Time-series data of the robot's joint angles and tool positions.
5. Intrinsics: Camera calibration parameters.
Target Output:
1. Out-of-FoV Organ Shape: The 3D point cloud for the organ outside the FoV at a specific frame, e.g., frame 10, 20, or 50.
Metrics:
1. Out-of-FoV Shape Chamfer Distance: Measures the overall average distance between the predicted organ shape and the ground truth point cloud to evaluate general shape similarity.
2. Out-of-FoV Shape Hausdorff Distance: Measures the maximum geometric error between the predicted organ shape and the real shape in hidden areas.

Join the Challenge!

Important Dates

May 1, 2026: Challenge Launch. Release of Training Data (DRENDS Public Dataset) and Development Kit.

July 10, 2026: Final Submission Deadline. Participants must submit their Docker containers for evaluation.

July 20, 2026: Evaluation Period. Organizers run submissions on the Hidden Test Set (New, unreleased sequences).

July 22, 2026: Announcement of Winners and Finalists.

August 1, 2026: BioRob 2026 Workshop. Winner announcements and presentations.

Awards

The top three winners in each task will be awarded monetary prizes. The prize structure for both Task 1 and Task 2 is as follows: USD 350 for first place, USD 200 for second place, and USD 100 for third place.

In addition to the monetary awards, winners will receive official certificates from the IEEE BioRob 2026 organizing committee.