Leveraging existing sparse point annotations
for benthic imagery dense segmentation
Computer Vision for Ecology (CV4E) Workshop - ECCV 2026
Embedded Files
Leveraging existing sparse point annotations
César Borja*, Breck A. McCollum, Jarrett E. Byrnes, Kenneth Sebens, Ana C Murillo*
Brief Description
Brief Description
The health of marine ecosystems is a critical indicator of global environmental change, yet the physical constraints of underwater observation and the intrinsic challenges of processing marine imagery severely limit the scalability of systematic monitoring. While recent vi- sual foundation models such as the Segment Anything Model (SAM) series show great promise, they still struggle with the fine-grained recog- nition required in these complex scenarios and still require expert supervision. Our work addresses this gap by bridging state-of-the-art foundation models with existing sparse supervision. Because historical benthic surveys are typically annotated with only a few sparse expert points per image, we utilize these legacy point-labels as visual prompts for SAM2. Our primary contribution is a novel mechanism to automatically identify which of these points are suitable, and which are actively harmful, when used for propagation. By filtering out unreliable points, we extract high- quality pseudo-ground-truth masks capable of training more accurate, fine-grained semantic segmentation models. We demonstrate the effec- tiveness of our approach on public benthic data and introduce a new, challenging benchmark featuring real-world sparse expert annotations, paving the way for scalable ecological analysis.
Overview of the proposed pipeline. Given an image with sparse point-labels, we propagate (1) each point-label i with SAM2. Using DINOv3 features, we calculate per-image class prototypes μc, and with them, we prune (2) the point-labels whose mask leak into other classes (i2), and trim (3) the patches that disagree with their mask’s class (p1, p4 and p5). The surviving masks are merged (4) into the final dense labels by ground-truth voting
Important links
Important links
Paper [pdf]
Code [TBD]
Dataset (images, acquisition protocol and annotations):
Sebens, K.P., E.J. Maney, and B.A. McCollum. 2026. Selected Photo Quadrats and Sparse Species Annotations from 2009 - 2023 at Subtidal Rock Walls around Halfway Rock, MA, USA ver 1. Environmental Data Initiative. https://doi.org/10.6073/pasta/d6ee65c4e31b14b7812ae37a336581b0 (Accessed 2026-08-12).
[train_split.txt] [test_split.txt]
Citation
Citation
Acknowledgments
Acknowledgments
This work was partially supported by grants AIA2025-163563-C31 and PID2024-159284NB-I00, funded by MCIN/AEI/10.13039/501100011033 and ERDF, and by DGA project T45_23R.
Page updated
Report abuse