Neural networks - online resource

posted 18 Jan 2019, 03:47 by Desmond M Ryan   [ updated 21 Jan 2019, 00:58 ]
For anybody interested in understanding neural networks there is an excellent (free) online book at:

The primary example in this book is to build a neural network which can recognise handwritten numbers (so digits 0,1,2,3,4,5,6,7,8,9) in images.

The simplest network has an input layer (image of number), one hidden layer (this is the decision maker), and an output layer (actually 10 outputs each corresponding to a digit, and in the simplest case is a 0 or 1).

The book discusses how the network is trained to recognise the digits (using a training set of 1000's of example images), and builds the network to increasing complexity (and performance), introducing convolution neural networks right up to deep learning.

The code for the number recognition can be downloaded from (just type this into firefox for download):

To run this code:

Log into ShARC, launch a qrshx session

cd /data/your_username
mkdir neural_networks
cd neural_networks

#upload to here
cd neural-networks-and-deep-learning-master/src

You need Python2.7 to run the code, so create a conda environment using:

#create the conda environment with Python 2.7 & numpy
module load apps/python/conda
conda create -n neural_net python=2.7 numpy
source activate neural_net

#launch python command line
#load the data loader helper
>>>import mnist_loader
#load the training, vaildation & test data
>>>training_data, validation_data, test_data = mnist_loader.load_data_wrapper()
#set up the network...input images are 28x28(=784) pixels, 30 hidden neurons and mini-batch size of 10
>>>import network
>>>net = network.Network([784, 30, 10])
#learn from the data over 30 epochs ((or cycles through the training data), mini-batch size 10 (number of random images from training set picked to calculate stochastic gradient descent) & a learning rate of 3.0
>>>net.SGD(training_data, 30, 10, 3.0, test_data=test_data)
You should see a result of ~9500 out 10000 meaning the trained network after epoch 30 can correctly identify 9500 images correctly out of the 10000 test images.

You can manipulate the training rate, mini-batch size, number of hidden neurons to see how it affects network performance.

The book continues to improve network performance by developing the model in complexity.

Good luck!.

Des Ryan