Current Research Projects

Topological Methods for Visual Recognition from Images and Video

We are interested in developing automated algorithms for visual recognition, i.e., recognizing an object from images of the object.  Applications range from robot visual systems to visual internet image search.  Systems that recognize an object from images work by comparing images of the object that is to be recognized to prior labeled images of objects, i.e., the training image database.  Because there are so many different objects, variability among the same objects (e.g., a person can appear in different articulations, with different clothing, etc...), and variations in the image formation process (e.g., lighting conditions, the vantage point of the camera, quantization level of the camera, and noise), the image database must naturally be very large.  However, for applications such as robot visual systems and visual search, the search through the database must take less than a fraction of a second and yet be very accurate for robot applications and quite accurate for visual search although some error can be tolerated. 

We are developing the mathematical methods to compare and match images of objects to the image database in a manner that is both fast and accurate.  We are not only interested matching single images to images in a database, but also multiple images (e.g. taken from various vantage points of an object) to databases of objects taken at multiple vantage points (as objects live in a three dimensional world).  We have recently made remarkable discoveries that will enable visual recognition for realistic applications - that only a sparse subset of an image is relevant for recognition.  The discovery enables reducing an image to a small fraction of the data without losing information for recognition, and also reducing the number of images in the databases.  The theory that we are developing uses techniques from computational topology, specifically, Morse theory, and we are extending Shannon's information theory to define information for visual signals as opposed to communication that Shannon's theory has been developed.  We are also extending these methods to video.

Collaborators: Stefano Soatto (CS/EE, UCLA) and Peter Petersen (Math, UCLA)

Some Publications:
Object Shape/Motion Analysis and Modeling for Video Editing / 3D Reconstruction

We are interested in developing semi-automated methods for visual object tracking from video. We are interested in developing tracking methods that delineate the outline and precise shape of the object(s) of interest.  An example application is automated rotoscoping for object animation in movies.  We are interested in creating tools for an interactive video editing software with emphasis on object level processing.  For example, a task could be to interactively give a rough outline of the object of interest in the first frame of the video and have the algorithm be able to automatically detect the precise boundary of the (deforming and moving) object in subsequent frames from which cut outs and editing on the object could take place.  The objects we are interested in are near to the video camera and display large articulations (e.g. a human moving) and non-rigidity, are taken under complex lighting conditions, and the objects can be partially occluded by other objects or self occlusions.  Because of the large variability of the objects and the specific applications that we are interested in, training data (a requirement of most visual tracking systems) is infeasible.

In order to enable interactive video editing at the object level, we are developing the mathematical methods to model moving and deforming objects that are projected into the imaging plane.  In particular, we are using methods of Riemannian geometry to model statistical variations of an object's shape, and developing the techniques to match the deforming object to time varying image data.  We are generalizing prediction, estimation, and filtering techniques such as Bayesian filtering to objects, which live in a high dimensional non-linear space (e.g., manifold), rather than to vectors for which traditional Bayesian estimation theory applies.

Collaborators: Stefano Soatto (CS/EE, UCLA), Andrea Mennucci (Scuola Normale Superiore), Anthony Yezzi (ECE, Georgia Tech)

Some Publications:

Medical Image Analysis for Disease Identification

We are interested in inferring and extracting (segmenting) anatomical structures of interest from medical imagery (such as magnetic resonance images (MRI), diffusion tensor (DT-MRI), and time-varying MRI), and developing algorithms to compare and match extracted structures among many subjects.  Such imagery is obtained for diagnosis purposes, and typically a medical expert must manually segment the image, and since the imagery is three dimensional (and possibly time-varying) the process is both time consuming, and prone to errors due to the fact that manual segmentations are done on two-dimensional slices while the structure lives in three dimensions.  Thus, we are interested in developing automated and semi-automated algorithms for structure extraction in specific application scenarios.  We are also developing quantitative methods for comparing three-dimensional structures and performing statistical analysis among the structures of patients in order to measure regions of variation and distortion of a structure from the rest of the population.  We are currently working on brain sub-structure analysis for classification of disease, and cardiac MRI segmentation and motion analysis for understanding myocardium function.

To enable these applications, we are developing the mathematical tools to register and match objects with high variations in shape, generalizing principal component type analysis to anatomical structures, registering and matching images, and deforming shapes to fit image data.

Collaborators: Allen Tannenbaum (ECE/BME, Georgia Tech), Ivo Dinov (LONI, UCLA), Byung-Woo Hong (Chung-Ang U, Korea)

Sample Publication: Tubular Surface Segmentation for Extracting Anatomical Structures from Medical Imagery (IEEE Transactions on Medical Imaging 2010)

Shape Reconstruction / Analysis from Electron Microscope Images for Water Treatment Applications

We are interested in developing automated methods for reconstructing nano-scale asymmetric membrane shape from field emission electron microscopy.  We are developing these methods to aid in the tuning of porous membranes for nano-filter construction and also for scientific studies that study the pore formation process in the construction of these membranes.  These filters are constructed by casting a solution on a substrate, and the chemical reaction forms pores in the substrate.  The membrane formed is cut with a laser beam and the layers are imaged using a microscope.  It is of interest to study quality of the pores, i.e., the porosity, to understand the quality of the filter and learn how to cast the solution to construct better quality filters.  Thus, we are developing methods to segment and reconstruct the shape of the membrane that has complex geometry and topology, and also quantitative methods to evaluate the porosity and quality of the mebrane from the reconstruction.  These nano-filters have the potential to be used for water treatment and desalination.

Collaborators: Suzana Nunes (Water Desalination Center, KAUST), Markus Hadwiger (GMSV, KAUST)