Learning Object Following for Small UAVs

Learning Object Following for Small UAVs using Vision-Language and Efficient Visual Flight Control.mp4

GradCAM showing human-biased attention
This work was acceptet at European Robotics Forum (2026) - Stavanger Norway

Learning Object Following for Small UAVs with Vision and Efficient Visual Flight Control

We introduce a vision-language-guided framework where a frozen model generates pseudo-labels to train a lightweight YOLOv8 student for markerless drone detection. Coupled with a depth-free IBVS controller operating solely on 2D image features, our system enables robust, real-time indoor UAV tracking with minimal manual supervision. Validated in classroom flights, it performs reliably across varied trajectories and supports deployment for surveillance, inspection, and, automation.

Main contributions:

Development of an indoor aerial dataset specifically curated for drone detection and tracking applications, addressing the scarcity of indoor UAV interaction data
Implementation of a reduced Image-Based Visual Servoing (IBVS) controller that operates exclusively on 2D image features without requiring explicit depth estimation, mitigating singularity issues inherent in traditional IBVS approaches
Design of an automated training pipeline incorporating self-supervised data annotation techniques for detection and segmentation model development, reducing manual annotation overhead

GradCAM demonstrating distributed drone attention in UAV-focused scenarios

Our results show effective indoor UAV tracking via combined dataset development and depth-decoupled visual servoing. A 98.6% automated labeling accuracy supports self-supervised annotation for domain-specific tasks, reducing manual effort. YOLOv8-nano offers the best real-time trade-off (0.97 precision/recall, 4.7 ms inference), while YOLOv8-small 49% latency increase yields minimal accuracy gain, underscoring onboard constraints. GradCAM reveals human bias in mixed scenarios, suggesting degraded performance in populated settings and validating our drone-to-drone indoor focus. The depth-decoupled IBVS controller outperforms PID baselines (89–92% vs. 68–71% centering), especially in high-curvature maneuvers. Limitations include dataset scale (14,400 frames), height-sensitive approximation errors, and reliance on continuous target visibility and sufficient texture for ellipse fitting.

Page updated

Google Sites

Report abuse