Embedded Files
Challenge 1: Continuous 3D Reconstruction While Eating
Background
Continuous 3D reconstruction from monocular video represents a frontier problem in computer vision, demanding algorithms that jointly reason about geometry and temporal coherence. Unlike static reconstruction, food objects in real-world manipulation videos undergo continuous rotation, deformation, occlusion, and breakage, leading to topological changes that challenge conventional assumptions of rigidity and appearance consistency.
Building on the 2025 Multi-Food 3D Reconstruction Challenge, the 2026 edition focuses on continuous, temporally coherent reconstruction of food during consumption. Participants will reconstruct evolving 3D food models from egocentric eating videos, where food items are partially occluded by hands and utensils and may leave the field of view. This task pushes algorithms beyond frame-wise reconstruction toward spatio-temporal reasoning that links motion, state change, and geometry. By evaluating methods under continuous deformation and topological variation, we aims to advance dynamic scene reconstruction, object pose estimation, and physics-aware modeling --- core directions for developing robust and generalizable 3D vision systems.
Description
This year's challenge will emphasize egocentric eating videos, where food items are partially occluded by hands and utensils and may leave the field of view. These changes are motivated to align the challenge more closely with real-world applications. In addition, same as last year, the removal of explicit physical references encourages the development of algorithms that can infer scale and orientation from implicit cues (such as plates, utensils, or common food items of known size) found in natural eating settings. This realistic setup not only enhances the challenge's relevance to practical applications in nutrition tracking and dietary assessment but promotes the creation of more intelligent and adaptive computer vision solutions.
To start, we recommend you use image to 3D generation model to create a mesh. Then use metric depth estimation model to calibrate the size. See rules in the kaggle page for the recommended external resources.
Evaluation Criteria
The evaluation will consist of a two-phase process focused on assessing the precision of the reconstructed 3D models in terms of both shape (3D structure) and portion size (volume). Results will be compared against reference models of the same food items captured using a 3D scanner.
In Phase I (Volume Accuracy), participants will submit a predicted before-eating volume and a predicted after-eating volume for each food item. The evaluation employs a composite score combining two complementary measures: (1) MAPE on the before-eating volume, assessing the accuracy of absolute portion size reconstruction, and (2) MAE on the percentage consumed, where the predicted consumption ratio is derived from the participant's own submitted volumes. This formulation rewards both accurate 3D reconstruction and internally consistent before/after estimation — recognizing that in real-world dietary assessment, the proportion of food consumed is often the most actionable clinical information. The final Phase I score is an equal-weighted average of both terms, with lower scores indicating better performance.
In Phase-II (Shape Accuracy), The top-ranking teams (3-5, depends on the total number of teams) from Phase I will proceed to Phase II, where they will submit complete 3D mesh files for each food item. We will use the L1 Chamfer Distance metric to evaluate the shape accuracy of the models. Our evaluation protocol follows the Chamfer Distance metric used in DTU benchmark.
The final winner will be determined by combining scores from both phases for all private food objects. Each team will receive a ranking (1–5) in each phase, and the final score will be a weighted sum
Phase I (Volume Accuracy): 55% weight
Phase II (Shape Accuracy): 45% weight
Important Dates
Challenge Open: March 22nd, 2026
Phase-I submission deadline: May 3rd, 2026
Phase-II 3D models submission deadline: May 4th, 2026
Phase-II transformation matrices submission deadline: May 10th, 2026
Announcement of Challenge Winners: May 14th, 2026
For detailed challenge instruction and regulations, please refer to the Kaggle page: https://www.kaggle.com/competitions/mtf-challenge-1-continuous-3-d-reconstruction-while-eating/overview
Page updated
Google Sites
Report abuse