Abstract

Photometric bundle adjustment (PBA) is widely used in estimating pose and geometry by assuming a Lambertian world. However, the assumption of uniform radiance is often violated as the non-Lambertian reflections are quite normal in the real-world environment. Therefore, most existing PBA-based works suffer from photometric inconsistencies. To fundamentally overcome the inconsistency problem, we propose a novel physically-based PBA method. More specifically, we introduce a physically-based weight concerning material, illumination, and light path. This weight adjusts for inconsistent photometric errors. We also design corresponding models for material estimation based on sequential images and illumination estimation based on point cloud. In addition, we establish the first SLAM-related dataset of non-Lambertian scenes with complete ground truth of illumination and material. Experiments demonstrate promising improvements in accuracy compared to the existing visual simultaneous localization and mapping methods. 

Materials Associated with the Submission

Supplementary Document: [link]

Dataset: [link]