Networking retinomorphic vision sensor with memristive crossbar for brain-inspired visual perception
Advised by Prof. Feng Miao at Nanjing UniversityÂ
Motivation
How to overcome the bottleneck that convolutional machine vision composed of an image sensor and processor suffers from high latency and large power consumption?
Implementation of convolutional neural netork with the retinomorphic vision sensor
The objective of this project is to achieve a reconfigurable vision sensor to simulate the function of Convolutional Neural Network (CNN) in image sensing, processing and classification. In this project, I constructed a Convolutional Neural Network (CNN) with MATLAB to help verify the efficiency and superiority of the vision sensor prototype based on a memristor array. The accuracy can achieve nearly 100% and the decent outcome shows that our retinomorphic vision system can successfully identify three specific letters(n,j and u).
Object tracking of neuromorphic vision system by using RNN
The box in the flow chart below represents the field of view defined by the retinomorphic sensor. By using the sensor, the cross edge in the field of view is extracted as its key feature. Once the cross is recognized by the ANN, the position information (Xn, Yn) of the cross at Tn is input into a trained RNN to achieve the object tracking. The trajectory of the cross measured by the retinomorphic vision system (green line with dots) is compared with the predicted trace by RNN (orange line with dots).
In this project, I programmed to develop Recurrent Neural Network (RNN) with MATLAB to successfully predict the location and trajectory of a moving cross and the mean squared error was less than 10^-2. The gif of the tracking process is shown below.
