Current Research Projects
Quickest Moving Object Detection
Motion is a key cue for biological visual systems, and we often detect objects when they move. Therefore, detecting changes in apparent motion in videos is a natural approach for detecting objects from video. However, motions of objects are often ambiguous to compute from limited data (few frames) due to distractions from other moving nuisances or other changes in the scene, and doing so would lead to false detections. Motion cues can be made less ambiguous by observing more data (frames), but this leads to increased computational cost. Since we are interested in real-time systems, this increased cost may not be tolerable. We are constructing algorithms that address the tradeoff between computational cost (delay after the object of interest moves) and false detections due to clutter or weak motion cues. Quickest Detection, building on optimal stopping theory, provides a framework for minimizing the delay in detecting changes in stochastic processes while constraining false detections to an acceptable tolerance, and we are building on this framework to address the problem of quickest moving object detection.
[Sample papers: arXiv 2016 Technical Report]
Causal Video Object Segmentation and Prediction With Sobolev Region Metrics
We are developing algorithms to label pixels of distinct objects in videos. This problem of object segmentation has importance in a variety of applications (e.g., designing a robot that picks up and manipulates an object, or in motion picture processing). Many of these applications require real-time operation, and thus we are aiming for real-time operation by constructing causal algorithms. The challenges are that objects exhibit significant geometric and appearance variability in video due to viewpoint, object deformation, illumination and occlusions. Even for rigid objects, deformations induced by change of viewpoint form an infinite dimensional group. Therefore, we are developing new computational and analytic tools for modeling objects as dense regions (to model shape) and dense radiance functions on regions (to model radiance). We are developing optimization tools on such models for segmentation, and we are developing prediction and filtering tools to estimate shape and radiance of objects across time. The basic mathematical structure to enable these computational tools is a metric, which quantifies shape changes, and we have been exploring Sobolev region metrics, which we have developed using methods of Riemannian geometry. Such metrics are natural for modeling objects, which tend to deform smoothly, because of their smoothness properties.
Shape-Tailored Scale Spaces and Invariant Descriptors
One of the greatest challenges in designing computer vision algorithms is the variability of appearances of the same object or material due to nuisances in image formation, for example, the geometric distortion across an image of plaid-patterned cloth due to perspective or wrinkles in the cloth, or the photometric variation due to lighting. To overcome this, invariant descriptors have been developed, which compute statistics (aggregated from local patches) that do not change with respect to nuisances. However, such statistics must be computed within objects or materials otherwise irrelevant data from other objects corrupt the descriptors. Since the purpose of descriptors is to perform higher-level tasks, e.g., segmentation, segmentation is not available. Therefore, we are generalizing such descriptors to be computed while performing higher level tasks (segmentation, matching). We call these Shape-Tailored Descriptors as they are tailored to the shape of the object. We are building on optimization theory and specifically optimization with partial differential equation (PDE) constraints to compute such descriptors efficiently, and use them in applications.
Fast Label: Efficient and Simplified Optimization for Multi-Label Problems
Many computer vision algorithms require labeling pixels in images with a discrete set of labels (e.g., segmentation problems) and such problems are often formulated as optimization problems.
Invariant and Selective Descriptors for Matching
Image correspondence (finding corresponding regions/parts/pixels between images) among images with large changes of viewpoint, occlusions, illumination change, and object deformations is a fundamental task in computer vision with a wide number of applications (segmentation of objects in video, 3D reconstruction, etc). Under such scenarios, registration methods based on optimization often fail since such methods make assumptions on the amount of displacement. While descriptor / detector methods are able to cope with large displacements, they often fail under large viewpoint change in non-planar scenes and only match sparse parts of the image. Such methods often fail since they are unable to capture the right tradeoff between invariance / covariance of the descriptor / detector and selectivity for matching. We are developing methods that are better able to address this tradeoff in the matching task by computing such invariances of regions during the matching process. We are building hierarchies of regions and hierarchies of invariant descriptors to perform matching in a coarse-to-fine manner for efficiently. To do this, we are building on methodology from group theory.
Physically Motivated Motion Estimation and Medical Image Registration / Segmentation
Image registration (finding pixel-wise correspondence in images) is a problem of wide importance in computer vision. The difficulty of the problem comes in part due to the aperture ambiguity, that is,
[Sample Papers: TMI 2014]
SurfCut: Free-Boundary Surface Extraction and Application to Seismic Images