Neural Networks (Computational Intelligence)

Neural Networks
Much current work in artificial intelligence is focused on neural networks (a form of computational intelligence). An artificial neural network in a computer attempts to mimic the actions of neural networks of the human brain.

Artificial neural network
A computer representation of knowledge that attempts to mimic the neural networks of the human brain.


Biological neural networks
A neuron is a single cell that conducts a chemically based electronic signal. The human brain contains billions of neurons connected into a network. A neuron is either excited or inhibited. An excited neuron conducts a strong signal; an inhibited neuron conducts a weak signal. A series of connected neurons forms a pathway.
A biological neuron has multiple input tentacles called dendrites and one primary output tentacle called an axon. The dendrites of one neuron pick up the signals from the axons of other neurons to form a neural network. The gap between  an axon and a dendrite is called a synapse. See figure below:


A neuron accepts multiple input signals and then controls the contribution of each signal based on the "importance' the corresponding synapse assigns to it. If enough of these weighted inpurt signals are strong, the neuron enters an excited state and produces a strong output signal.
Neurons fire, or pulse, up to 1000 times per second so the pathways along the neural nets are in a constant state of flux. As we learn new things, strong neural pathways form in our brains.


Artificial Neural Networks
Neural networks are trained by learning patterns using trial and error, just as the brain does.
There is considerable debate in the AI community about the future of neural nets. Some see neural nets as playing only a limited role in AI; others expect them to overtake the current rule-based approach.

Neural networks are typically organized in layers. Layers are made up of a number of interconnected 'nodes' which contain an 'activation function'. Patterns are presented to the network via the 'input layer', which communicates to one or more 'hidden layers' where the actual processing is done via a system of weighted 'connections'. The hidden layers then link to an 'output layer' where the answer is output as shown in the graphic below.


Each processing element (equivalent to a neuron) has a numeric threshold value. The element compares the effective weight to this threshold value. If the effective weight exceeds the threshold, the unit produces an output value of 1. If it does not exceed the threshold it produces an output of 0.

The process of adjusting the weights an threshold values in a neural net is called training. A neural net can be trained to produce whatever results are required.

                                                          Comparison of Biologics vs Neural network


Imagine to find the cat in the photograph below using a neural network using one output value per pixel.

Our goal is to produce an output value of 1 for every pixel that contributes to the image of the cat, and to produce  a zero if it does not. The input values for the network could come from some representation of the colour of the pixels. We then train the network using multiple pictures containing cats, reinforcing weights and thresholds that lead to the desired (correct) output.

Uses of neural networks include: hand writing recognition, data mining, spam filtering, mortgages, detection of plastic explosives at airports.


Below is a program written using a version of Python for Optical Character recognition using a neural network

Neural Networks Summary


Additional resources

The idea of teaching a robot very basic things, like avoiding objects or following another person, seems like it should be very simple, but it is actually quite complicated. In order to learn about the process of "training" a machine, we can use a game called Nero (Neural Network, Machine Learning game) which allows you to actually train "droid" players in a game world in order to attack, defend or otherwise complete missions.

Assignment
ITGS May 2009 Paper 1 Q4: Artificial Intelligence (due 15 January 2014)           Markscheme

Subpages (1): Neural Nets in Python