The pointclouds above were generated by the same model, that has never seen any of these datasets, and without groundtruth scale alignment.
Abstract. Monocular depth estimation is scale-ambiguous, and thus requires scale supervision to produce metric predictions. Even so, the resulting models will be geometry-specific, with learned scales that cannot be directly transferred across domains. Because of that, recent works focus instead on relative depth, eschewing scale in favor of improved up-to-scale zero-shot transfer. In this work we introduce \Acronym, a novel monocular depth estimation framework capable of predicting metric scale for arbitrary test images from different domains and camera parameters. This is achieved by (i) the use of input-level geometric embeddings that enable the network to learn a scale prior over objects; and (ii) decoupling the encoder and decoder stages, via a variational latent representation that is conditioned on single frame information. We evaluated ZeroDepth targeting both outdoor (KITTI, DDAD, nuScenes) and indoor (NYUv2) benchmarks, and achieved a new state-of-the-art in both settings using the same pre-trained model, outperforming methods that train on in-domain data and require test-time scaling to produce metrically accurate estimates.
Contributions:
We introduce ZeroDepth, a novel variational monocular depth estimation framework capable of transferring metrically accurate predictions across datasets with different camera geometries.
We propose a series of encoder-level data augmentation techniques aimed at improving the robustness of our proposed framework, addressing both the appearance and geometric domain gaps.
As a result, ZeroDepth achieves state-of-the-art zero-shot transfer in both outdoor (KITTI, DDAD, nuScenes) and indoor (NYUv2) benchmarks, outperforming methods that require in-domain training images and test-time ground truth scale alignment.
