My Research Work
 

Research and technical work

Research during PhD at University of Manchester

My PhD work consisted of building software models for neural systems which are scalable, can store and retrieve sequences with state information. Below is a brief description: 

I am concerned mainly with rank-ordered N of M coded Kanerva (Sparse Distributed) memories. The Elman-Jordan network model is used to incorporate state information. The basic model is a two-layered neural network with recurrent connections. The idea is to decode the input pattern into high dimensional space, where it is much more likely to be linearly seperable. Hebbian learning takes place in only one of the two layers, while the weights are fixed in the other layer. The second of the two layers is a correlation matrix memory. I am currently working on time abstracted neural models which learn 'on the fly', that is, in a single pass. Such neural structures are claimed to have properties similar to the cerebellum in the brain.

The novelty of the model is due to its scalability, similarity metrics, temporal properties due to the use of rank order coding, and speed of learning, which may be done in real time. The training and recall phases can be either seperated or done together. A generic finite state machine may be implemented by this network model. Multiple units of the two-layer model may be used to give additional flexibility. Hebbian learning is highly localised and this makes it easier to implement in hardware. Retrieval of correct output data on slightly errorenous input data, storing of sequences, etc are properties inherent in the neural memory model.

The models are being implemented and tested in hardware, on an EPXA1 development board using Excalibur EPXA1 devices. Scalability and speedup due to hardware are the major motivations for modelling in hardware. Locally synchronous, globally asynchronous model is to be used for the hardware.

Potential applications of ther work include pattern (like character/image) recognition, time series prediction, data classification or clustering, robot navigation, motif discovery, genome mapping in DNA sequences, learning of tunes etc.

The ultimate goal of the hardware project is to create scalable neural chips using FPGA logic, which can be joined in a modular fashion to build a large-scale neural network, with the inter-chip communication limited to reporting of events(firing of the neuron). Altera chips are to be used, along with a hardware development board to design the functioning. Thus the project would also involve, at a later time, things about the high-level interface for programming the chip, the primitive communication protocol to be used for inter-chip communication, a primitive operating system for the microprocessor, etc. The Altera chips have a microprocessor and RAM memory to store the information. The software model, once it is developed, is to be transferred to the chips using Altera Excalibur EPXA1 Hardware Development Board (which uses an ARM processor along with APEX FPGA) and the Altera Quartus II Software. Verilog is the dominant HDL to be used.

I am also interested in neural models for storing sequences of groups of neurons firing seperated in time. Implementing the above ideas in real time involves resolving the synchronisation problems between different waves of firing neurons, and maintaining their seperation as well as their order. Also, implementing the N of M model described above in real pulse-coded neurons involves getting only a specific fixed number of neurons with the highest activations to fire, the time of firing being in order of their activations. Use of/Comparision with other neural architectures/algorithms such as self organising maps, genetic algorithms or support vector machines may also be investigated. Modelling of non-linear functions is a subject of interest.

Heres an overview of the sequence machine, in layman's terms:

Definition: What?

I have created a sequence machine using spiking neurons. Let me explain what that really means: a sequence macine is something that can "recognise" sequences of patterns. I am defining it as a deterministic automaton that can learn and predict sequences of patterns. Now the patterns can be anything such as notes in a tune or english alphabet or a sequence of images as in a movie.

The main thing about a sequence is time: it is a collection of patterns in time. Now in our heads we also represent (store and retrieve) time, eg when we store a song and can replay it. Time is stored spatially, as the only means we have are the interconnected neurons in the brain, and the connections are what store the information.

Motivation: Why?

Its an intriguing question as to how biology does it, which I seek to understand through modelling. Also, if you have read a book by Jeff Hawkins (On Intelligence), the neocortex part of the human brain (in the forebrain, it is responsible for higher functions such as intelligence) works exactly this way: as a sequence memory and predictor. Its intuitive and easy to understand: we make predictions all the time witrh our senses: for example we predict every morning how our office is going to look like.

As you may or may not know, all information in the brain is transmitted as spikes or action potentials, which are small electrical pulses. The beautiful scenery, the soothing music, the delicious dinner, etc all information gets transmitted to the brain in the language of spikes. And likewise for the brain commands to the motor organs (such as hands and legs) to do things such as move. If you want to know how that happens, you can see Fred Rieke's book called Spikes (MIT Press). Similarly, my sequence machine also first converts the patterns constituting a sequence as spikes following a specific code (which I will explain), and then stores the patterns in the connections of artificial neurons in the brain. The "spikes following a specific code" is in reality also a sequence of spikes, so I am storing a sequence of sequences of spikes in my artificial neural network, which in a way is like a mathematical abstracted model of the neurons in the brain. Also the interconnected neurons in this artificial neural network which I use to store the sequences spatially (just as in a real brain) in their connection strengths, can adapt themselves or 'learn' from the sequences that are input to them, and that is how they can make the right predictions.

Method or approach: How?

The approach is broadly based on how biology does things. It is a model of short and long term memory, and also a theory on how short term memory gets transformed into long term memory gradually. Besides, the network is robust (to a degree) to input error. There are some restrictions, however, on how much time the inputs must be separated by, so that the neural network may not confuse between two patterns.

Technically speaking, I have used an associative memory to store the sequences. The spiking neural model is a variant of the standard leaky integrate and fire model (read a book by Mass called pulsed neural networks, MIT Press, to know about such models. The coding I have used to encode any pattern as a burst of spikes is called rank ordered N-M code, which comes from rank roder codes used by Thorpe (see article by Simon Thorpe in nature) and N-M codes used widely in asynchronous logic and also popularised by a paper by Furber in neural networks.

Here's another question: why bother? Why should you be interested in this? Well, its a piece of research in an interesting field, memories. And also it might be possible (at a later stage) to build some interesting applications with this kind of model. A guy I met at a conference I recently went to, called Ander Maurer, trained a robot to see and reproduce guestures using a model quite similar to mine. And well, for those who have read Hawkins' book, this model is a concrete example (in a way) of the kind of things the book speaks about, though of course the structure of the network is nothing like the neocortex. Besides, there are some interesting problems I have attempted to treat by the side, such as how to develop a good capacity robust associative memory, the dynamics of complex systems involving layers of spiking neurons with feedback, how to best store time as space, how to have a distance metric between codes when the code you use to encode patterns does not have an inherent concept of distance, and thus what does the distance notion actually mean, etc.

Research Project Group members

The current members of the neural Network Fire Project Team are:

  • Steve Furber(Supervisor and head)
  • Jonathan Shapiro(Joint Supervisor of Joy Bose)
  • Mike Cumpstey
  • Hamed Shihidipour
  • Basabdutta Sen
  • Joy Bose

Previous members:

  • Peter Marshall
  • Lihua Ren

Please mail me if you have any comments/ ideas/ suggestions/ objections/ questions regarding my work.

Papers

(1) An associative memory for the on-line recognition and prediction of temporal sequences. in IJCNN 2005, Montreal, Canada

(2) A spiking neural sparse distributed memory implementation for learning and predicting temporal sequences. In ICANN 2005, Warsaw, Poland

(3) A system for transmitting a coherent burst of activity through a network of spiking neurons. Published in WIRN 2005, Vietri Sul Mare, Italy

(4) A neural network for recognition and prediction of temporal sequences of patterns , published in PREP 2005, Lancaster University

(5) A sequence machine built with an asynchronous spiking neural network in 17th UK Asynchronous Forum, Southampton, Sep 2005 

(6) An asynchronous spiking neural network which can learn temporal sequences in 18th UK Asynchronous Forum, Newcastle, Sep 2006 

(7) Sparse Distributed Memory using Rank Order Neural Codes (Co-authored with Steve Furber, Gavin Brown, Mike Cumpstey and Jon Shapiro) in IEEE TNN, May 2007

Paper Drafts

Draft Paper: an online predictive sequence machine using rank order coded predictive spiking neurons

PhD Dissertation

Download my PhD dissertation titled "Engineering a sequence machine out of spiking neurons employing rank order codes" (5.23 MB, 216 pages)

Research presentations

APT Group meeting, 18 June 2003 slides

Colamn meeting June 2006 Slides

APT group meeting, July 2006 slides

Colamn meeting, March 2006 slides (only 4 slides are my work)

Presentation to AI group: 10 Nov 2004

Presentation to APT Group, 28 Oct 2004

APT Group Presentation, Jan 2003

Presentation to CS 700 research symposium, December 2002

Presentation to UK async forum in Newcastle, September 2006

Memory capacity experiments, September 2003

NIPS report, December 2003

NIPS 2003: notes

Workshops attended in NIPS 03

RNNaisance Workshop, NIPS 2003

NN conferences list, prepared 2004

PhD Transfer Interview 2003 presentation

Prep conference April 2005 presentation (for poster)

Research Students Symposium, 3 Nov 2004

ICANN 2005 presentation, September 2005, Warsaw, Poland

Poster for IJCNN 2005, 25 August 2005, Montreal Canada

WIRN conference June 6 2005, Slides presentation

WIRN 2005 Poster, June 6 2005

End of year 2 presentation, August-September 2004, and end of year 2 report

PhD viva questions: preparation

PhD Transfer report / MPhil thesis draft

A scalable sparse distributed neural memory model

A typical experiment 

See the page A Complete Research Experiment 

My coding page

Various perspectives on the research

Writing a paper

Research and technical papers written during employment at Samsung R&D Institute India- Bangalore

My work in Samsung India consisted mainly of different aspects of mobile web browsers, TV and multiple screen scenarios, wearable devices and EEG sensors and assorted topics in mobile devices and connectivity. The focus is on improving the overall user experience. 

Papers Published/ accepted

(1) Gesture based sensor encryption (Co-authored with Tasleem Arif) in Second World Congress on information and Communication Technologies (WICT2012), Nov 2012

(2) A solution for a mobile computing device along with supporting infrastructure for the needs of illiterate users in rural areas (Co-authored with Dipin K P) published in IEEE INDICON, Dec 2012

(3) A two way emergency medical monitoring system with a computing device fitted on a rotating holder (Co-authored with Dipin K P, Srimanu P N and Vivek V G) to be published in IEEE INDICON, Dec 2012


(5) Contextual adaptive user interface for Android devices, accepted for publication in IEEE INDICON 2013, IIT Bombay, India, December 2013

(6) Context based adaptation of application icons in mobile computing devices, accepted for publication in WICT 2013, Hanoi, Vietnam, December 2013

(7) Browser with clustering of web documents, accepted for publication in ADCONS 2013, NIT Surathkal, December 2013

(8) A framework for text summarization in mobile web browsers, accepted for publication in IEEE ICCIC Conference, Madurai, India, Jan 2014

(9) Reducing power consumption in Graphics intensive android applications, published in COMSNETS, Bangalore, India Jan 2014

(10) Reducing Power consumption in Android applications. Published in AICC Conference in Gurgaon, Feb 2014. 

(11) User data management by tabs during a browsing session, Published in DICTAP, Thailand, May 2014

(12) Browser with Tab Clustering,  Published in APWiMob in Bali, Indonesia, Aug 2014

(13) Responsive and Adaptive Rendering, Published in IEEE SEWAD conference in Greater Noida, September 2014

(14) Hybrid authentication mode based on Fingerprint, published in IEEE SEWAD conference in Greater Noida, September 2014

(15) Context based content mode, published in ICACCI conference in Greater Noida, September 2014

(16) A kiosk based model for employment generation in rural areas, published in IEEE Global Humanitarian technology conference South Asian Satellite, GHTC SAS in Kovalam, September 2014


(18) Multicast routing with load balancing using Amazon Web service, IEEE INDICON in Pune, December 2014

(19) TV Remote control via wearable smart watch device, IEEE INDICON in Pune, December 2014



(22) Dynamic Website Specific Browser menus, in IACC 2015 in Bangalore in June 2015

(23) A Hands Free Browser Using EEG and Voice Inputs, selected in International Joint Conference of Neural Networks IJCNN 2015 in Killarney, Ireland, July 2015

(24) EEG Based Detection of Area of Interest in a Web Page, selected in ICACCI Special Session on Learning and Cognitive Modeling to Assistive Technologies , Cochin, India, August 2015

Patents filed in India patent office

Patents filed along with co inventors

System and method for advanced communication device security with tracking using cloud infrastructure
India 3595/CHE/2012
Filed August 30, 2012
Summary: This talks of a foolproof way to locate a stolen device by using the hardware Ids and track the device using a cloud server.

System and method for real time cloud assisted collaborative browsing across different user agents
India 3720/CHE/2012
Filed September 7, 2012
Summary: This talks of user agent independent collaborative browsing by using different methods to translate events across different user agents.

System and method for adapting the user interface based on pressure profiles generated by user interaction
India 4418/CHE/2012
Filed October 23, 2012
Summary: This talks of having pressure sensors on the sides and back of a device to have novel pressure based gestures, and also a novel method of identifying the pressure gestures. 

Touch-based gestures on tagged images on mobile devices
India 492/CHE/2013
Filed February 4, 2013
Summary: This talks of having contacting a group of tagged users in a photo via means such as call or email by having specific touch gestures for the same. 
India 618/CHE/2013
Filed February 13, 2013
Summary: This talks of a cloud server assisted remote download operation where the user can remotely control all aspects of the download from a mobile device
Abstract: A method and system that enables a user to initiate remote content download using a browser on a communication device is disclosed. The user can select one or more remote devices onto which the download is triggered. The method provides the user with plurality of user controls such as scheduling, prioritizing, status tracking and controlling of the download operations. The method enables the user to choose best cost effective remote device by providing the user with list of remote devices along with their network information of the remote device. The method enables the server to cache the remote download requests if the remote devices are currently inactive (unavailable). The method enables the user to view status and control the download requests from any browser of any of the remote devices or communication device connected to the server.
India 3996/CHE/2013
Filed September 5, 2013
Summary: This talks of assisting amateur photographers to set the zoom level automatically to optimal conditions, based on the actual picture composition, using commonly used professional photography techniques. 
Abstract: A method and system that enables a user to initiate remote content download using a browser on a communication device is disclosed. The user can select one or more remote devices onto which the download is triggered. The method provides the user with plurality of user controls such as scheduling, prioritizing, status tracking and controlling of the download operations. The method enables the user to choose best cost effective remote device by providing the user with list of remote devices along with their network information of the remote device. The method enables the server to cache the remote download requests if the remote devices are currently inactive (unavailable). The method enables the user to view status and control the download requests from any browser of any of the remote devices or communication device connected to the server.

A system and method to browse efficiently using tabs or windows on small screen devices
India 4916/CHE/2013
Filed October 31, 2013
Summary: This talks of a tabbed browsing case where the user doesn't have to leave the current browsing context while the tab opens in the background. 

METHOD AND APPARATUS FOR DISPLAYING DYNAMIC DATA CONTENTS OF A WEBPAGE ON A CLIENT DEVICE
India 524/CHE/2014
Filed February 4, 2014
Summary: This talks of extracting relevant information during the loading time from currently browsed web pages and displaying it in the address bar or in preview. 

System and method for suggesting URLs in web browsers
India 3612/CHE/2013
Filed February 5, 2014
Summary: This talks of methods to compute and suggest URLs to browse to the user on a mobile device without using any external server, based on the user's browsing history stored on the device. 

AN APPARATUS AND METHOD FOR PROVIDING POWER MANAGEMENT IN MOBILE DEVICES
India 843/CHE/2014
Filed February 20, 2014
Summary: This talks of having a secondary micro controller to control device actions in the low power case, so the user can still use the device for some essential functions. 

Automatically setting zoom level of camera lens using detected faces in preview
India 1091/CHE/2014
Filed March 4, 2014

Method and system for generating graphical transcript of a user during an interactive session
India 6328/CHE/2014
Filed December 22, 2014
Summary: This talks of generating an EEG based graphical transcript of the emotions of the user during a chat session, which other users can see, along with a lie detector. 

A method for optimizing access to application using wearable device in communication with electronic device
India 6610/CHE/2014
Filed December 30, 2014
Summary: This talks of various ways in which a wearable device can be used as a companion to complement the functions and applications on the main mobile device. 

Method and system of providing a web feed in a uniform resource identifier
India 6668/CHE/2014
Filed December 29, 2014
Summary: This talks of methods to access, extract and store RSS feeds in the device from any visited URL. 

METHOD AND SYSTEM FOR PROVIDING DYNAMICALLY CUSTOMIZED WEB PUSH MESSAGES IN A WIRELESS NETWORK
India 3367/CHE/2014
Filed April 23, 2015
Summary: This talks of an architecture, system and use cases to enable the gateway server to dynamically customize web push messages as per the user, device statistics and so on. 

Patents filed in USPTO
https://www.google.com/patents/US20140223345
Filing Date February 3 2014
Provided are a method of initiating communication in a computing device including a touch sensitive display and the computing device. The method includes detecting a touch gesture that is performed on at least one tagged object included in an image displayed on the touch sensitive display and initiating, in response to the detection of the touch gesture, communication to at least one individual...


US 20150121257 A1
http://www.freepatentsonline.com/y2015/0121257.html
Filing Date October 31 2014
A method and apparatus that enables a user to trigger a Uniform Resource Locator (URL) load requests on one or more background tabs while displaying the current browsing webpage on the foreground tab, is provided. The method includes displaying a browsing webpage on a foreground tab of the browser, displaying a plurality of address spaces for entering URLs on the foreground tab of the browser, receiving a first URL in a first of the plurality of address spaces, opening a first background tab for the received first URL without diverting from the displayed current browsing webpage on the foreground tab, and issuing a webpage load request on the first background tab for the received first URL. The method eliminates the need to switch to a new tab to trigger one or more URL load requests in new tabs and enhances the browsing experience of the user.