neuralnetworks

Neural Networks

Table of Contents

• 1. Feedforward network

     1. Back-Propagation Algorithm

     2. Recursive least Square

     3. Levenberg-marquardt

  2. Radial-Basis functions

  3. Hopfield network

1. Kohonen Self-Organizing Map

   1. Recurrent Neural Network

   2. Real time recurrent learning (RTRL) algorithm

Kohonen's Self Organizing Map (KSOM)

KSOM can be used to capture the topological map underlying the input data space. KSOM is quite useful to interpret data where input-output relationship is not known. KSOM can also be used to extract features where a large amount of input-data is available.

KSOM Learning comprises of following steps:

• 1. Form a 2-D or 3-D or higher dimensional lattice where each node represents a neuron. Each such node is assigned a weight vector of same dimension as that of input neuron. Weights are randomly initialized in the input space and these weights represent the neuron location in the input space.

  2. Input data is presented to the network one by one and the neuron with minimum distance from this input is declared a winner.

  3. Weights of this winner neuron as well as its neighbors (defined by a neighborhood function) are update so that they move closer towards the input neuron.

  4. Steps 2-3 are repeated for a large number of times.

Click here to download the program which implements a KSOM algorithm where a 2D lattice is used to learn the sinusoidal relationship between input and output. In the adjacent figure, the red dots represent initial location of neuron postions. As the training proceeds, these neurons organize themselves so as the capture the topological relationship hidden in the input data. This code was contributed by Balaji Uggirala, a PG student at my lab.

Real Time Recurrent Learning (RTRL)