Nonlinear Line Attractor Network 

Nonlinear Line Attractor for Learning Complex Manifolds of Visual Perception

The human brain processes enormous volumes of high-dimensional data for everyday perception. To humans, a picture is worth a thousand words, but to a machine, it is just a seemingly random array of numbers. Although machines are very fast and efficient, they are vastly inferior to humans for everyday information processing. Algorithms that mimic the way the human brain computes and learns may be the solution. In this paper we present a theoretical model based on the observation that images of similar visual perceptions reside in a complex manifold in a low-dimensional image space. The perceived features are often highly structured and hidden in a complex set of relationships or high-dimensional abstractions. To model the pattern manifold, we present a novel learning algorithm using a recurrent neural network.

The brain memorizes information using a dynamical system made of interconnected neurons. Retrieval of information is accomplished in an associative sense. It starts from an arbitrary state that might be an encoded representation of a visual image and converges to another state that is stable. The stable state is what the brain remembers. In designing a recurrent neural network, it is usually of prime importance to guarantee the convergence in the dynamics of the network. We propose to modify this picture: if the brain remembers by converging to the state representing familiar patterns, it should also diverge from such states when presented with an unknown encoded representation of a visual image belonging to a different category. That is, the identification of an instability mode is an indication that a presented pattern is far away from any stored pattern and therefore cannot be associated with current memories. These properties can be used to circumvent the plasticity-stability dilemma by using the fluctuating mode as an indicator to create new states. We capture this behavior using a novel neural architecture and learning algorithm, in which the system performs self-organization utilizing a stability mode and an instability mode for the dynamical system.

Based on the above observation we developed a self-organizing line attractor, which is capable of generating new lines in the feature space to learn unrecognized patterns. Experiments performed on various face lighting variant, pose variant and expression variant databases for face recognition have shown that the proposed nonlinear line attractor is able to successfully identify the individuals and it provided better recognition rate when compared to the state of the art face recognition techniques. These results show that the proposed model is able to create nonlinear manifolds in a multidimensional feature space to distinguish complex patterns.

Publications

• Ming-Jung Seow, Ann Theja Alex, and Vijayan K. Asari, “Learning embedded lines of attraction by self-organization for pose and expression invariant face recognition,” SPIE Journal of Optical Engineering, vol. 51, no. 10, pp. 107201: 1-8, October 2012.

• Ming-Jung Seow, Vijayan K. Asari, and Adam Livingston, “Learning as a nonlinear line of attraction in a recurrent neural network,” Neural Computing and Applications, vol. 19, no. 2, pp. 337-342, March 2010.

• Ming-Jung Seow and Vijayan K. Asari, “Towards representation of a perceptual color manifold using associative memory for color constancy,” Neural Networks, The Official Journal of the International Neural Network Society, European Neural Network Society & Japanese Neural Network Society, vol. 22, pp. 91-99, January 2009.

• Ming-Jung Seow and K. Vijayan Asari, “Ratio rule and homomorphic filter for enhancement of digital color image,” Journal of Neurocomputing, vol. 69, no. 7-9, pp. 954-958, March 2006.

• Ming-Jung Seow and K. Vijayan Asari, “Recurrent neural network as a linear attractor for pattern association,” IEEE Transactions on Neural Networks, vol. 17, no. 1, pp. 246-250, January 2006.

• Ming-Jung Seow and K. Vijayan Asari, “Color characterization and balancing by a nonlinear line attractor network for image enhancement,” Neural Processing Letters, vol. 22, no. 3, pp. 291-309, December 2005.

• Ming-Jung Seow and K. Vijayan Asari, "Learning using distance based training algorithm for pattern recognition," Pattern Recognition Letters, vol. 25, no. 2, pp. 189-196, January 2004.