Recent Projects

This work addresses the problem of planning the camera trajectory around the object to have a better 3D model. The next best views from a set of candidate poses are selected by maximizing a utility function based on information gain and movement costs. The candidate poses are defined by the robot dynamics and environmental constraints. Planning the views depends on the sensor modality and the definition of information. For this reason the framework is required to be highly modular to allow it to be used in different applications.

PHENOM features are texture independent interest points extracted from surface normal maps recovered by Photometric Stereo. Such sparse matching will help to register high detail 3D surfaces reconstructed from multiple view images. Moreover, the geometric constraints imposed by multiple views can be utilized to correct the geometric ambiguity in photometric reconstruction. Unlike standard intensity-based interest point detectors, such as SIFT, our features represent local surface geometries and do not rely on the observed image texture. Comparison against texture-based interest points shows that the proposed features based on normal maps perform effectively.

In this project we addressed the problem of simultaneous estimation of the vehicle ego-motion and motions of multiple moving objects in the scene—called eoru-motions—through a monocular camera in urban environments. Conventional localization and mapping techniques (e.g. Visual Odometry and SLAM) can only estimate the ego-motion of the vehicle. In this project, we have introduced a framework which exploits the constraints imposed by urban environments and by the vehicle kinematics to estimate multiple relative motions in addition to the camera ego-motion. The method is based on projective factorization of the multiple-trajectory matrix. Without a priori knowledge of the number of motions, several motion segmentation hypotheses are generated in a sequential RANSAC scheme. All the hypotheses are evaluated and the one with the smallest reprojection error is selected. The method is evaluated on popular street-level image datasets collected in urban environments.

LightPanel:

LightPanel is a mobile platform for dense 3D modelling developed in this project. The platform is an active image acquisition setup assisted with a set of light sources and a distance sensor. The hardware setup is designed for being mounted on a mobile robot which is remotely driven to create accurate dense 3D models from out-of-reach objects. For this reason, the object is actively illuminated by the imaging setup and Photometric Stereo is used to recover the dense 3D model. This image acquisition setup is built to investigate the opportunities of using Photometric Stereo for robotics applications and its practical challenges under uncontrolled lighting conditions.

Photo-Geometric Dense Reconstruction:

In this work, the problem of dense 3D reconstruction from multiple view images subject to strong lighting variations is addressed. In this regard, a new piecewise framework is proposed to explicitly take into account the change of illumination across several wide-baseline images. Unlike multi-view stereo and multiview photometric stereo methods, this pipeline deals with wide-baseline images that are uncalibrated, in terms of both camera parameters and lighting conditions. Such a scenario is meant to avoid use of any specific imaging setup and provide a tool for normal users without any expertise. Deals with such an unconstrained setting, a coarse-to-fine approach is used, in which a coarse mesh is first created using a set of geometric constraints and, then, fine details are recovered by exploiting photometric properties of the scene. Augmenting the fine details on the coarse mesh is done via a final optimization step. Although the method does not provide a generic solution for multi-view photometric stereo problem but it relaxes several common assumptions of this problem. The approach scales very well in size given its piecewise nature, dealing with large scale optimization and with severe missing data.