Other projects‎ > ‎

Face Recognition in Video


From sensing a visual stimulus 
(by eye & video-camera) 
to recognizing a face observed 
(in visual cortex  & Perceptual Vision System)


Will computers (man-made perception systems) be ever as efficient in recognizing what they see as are humans?

This was the question which drove us to look more closely into the way perception is done in  biological systems. It was realized then that not only  the  results from  Computer Vision and Pattern Recognition, commonly used by computer scientists working in the area,  should be used for this problem, but also the results obtained in Neurobiology.

We are still far from modeling  the entire brain (or visual cortex), which will unlikely ever happen, but some advances have been already made, most important  of which are 

  1. Setting  a distributed, non Von-Neumann, neural-network based paradigm for video processing and memorization, which allows one to incorporate the visual attention and other visual perception mechanisms  performed in the human brain , and 
  2. Establishing the framework for testing the performance of the thus developed "mini models of brains" - the way we call  our Facial Video Memory system.

One of the major advantages of the first result is that it allows one to deal with the continuous  flow of video images, as they are in video. -  Rather than storing an unlimited number of individual video frames, the approach uses the incoming flow of data to continuously tune the synaptic connections of a multi-connected neural network. This is analogous to the way associative memorization  is performed in visual cortex. Then, when a new video stimulus is presented to the retina, the neural network converges to state which is best described by the past seen/learnt experience. 

The second result is related to using video sequences, rather than individual video frames, which is in accordance with the developed paradigm. As claimed in our papers, high resolution is not needed for video-based recognition, as it  is not the way it is in biological systems and may, in fact, result in  overloading the problem. Hence,  our video sequences are 160x120, taken with an off-the-shelf webcam, the face of person in video occupying 1/16 - 1/4 of an image.

Direct applications of this technology are in:

  • Face Recognition for Security: Surveillance, Tracking and Backtracking, Biometrics
  • Face Recognition for Computer-Human Interaction: Perceptual User Interfaces, Nouse
  • Face Recognition for Video annotation and Games
  • General object memorization and recognition in low-quality video data

Where to go from here