Reading Handwritten Numbers - Multi-class Classification

In this example, we'll use Neurons to train a model to identify the correct digit (0-9) from a 28x28 pixel image of the hand drawn digit.

This was inspired by what 3blue1brown did in his video explaining neural networks.

For this example, we will be using the MNIST dataset, which is a dataset developed by the National Institute of Standards and Technology (NIST) for evaluating the performance of machine learning models in classifying handwritten digits.

The dataset consists of 28x28 pixel black and white images of the digits. Here are some examples:

We'll have to find a way of converting the data in these images to numerical values, since Neurons only accepts numbers. The way we do this is by converting each pixel to a gray scale between 0 and 1: 0 is a completely black pixel, and 1 is completely white.

Here's how it works:

So we now have 28x28=784 values between 0 and 1, per image, that we will use as inputs for our model. As for the output, in the case above it would be the digit 5. Since there are 10 possible output classes, we'll use One-hot Encoding (check the Dataset Structure section in the Classifying Iris Flowers example to know what this means).

These are the columns in every sample (row) of data:

• 0-783: Gray scale value of each pixel (between 0 and 1)

• 784-793: The digit written in the image (in One-hot Encoding)

The training dataset consists of 60,000 images, and the testing dataset has 10,000. At 794 numerical values per sample, this is pretty big data we're dealing with!

• Open Neurons. As the 'Train File', select 'digits_data_train.txt' inside the 'Data' folder.

• Select columns 0-783 as Input, these are the gray scale values of the pixels.

• Select columns 784-793 as Output, these are the digits corresponding to the images, One-hot encoded.

• Press 'File'->'New Model' or just press 'Ctrl + N' to create the model. We'll name it 'Digits_model'.

Note: After pressing 'New Model', Neurons will take a bit of time loading, and it might appear that it isn't responding. Don't worry, it'll work after a couple seconds. This happens because the dataset is loaded to check if it's valid, and being a huge dataset, it takes some time.

Here's how your screen should look:

We have 784 inputs in our model. That is a lot. The Input layer's 8 neurons aren't enough to successfully analyse all this data.

• Double click the Input layer. Change neurons to 784. Press 'Save Edit'.

• Double click 'Layer_1'. Change neurons to 500. Press 'Save Edit'.

• Double click 'Output'. Change Activation Function to 'SoftMax'. Press 'Save Edit'. The outputs are in binary, so we chose an activation function that outputs between 0 and 1.

This should give us enough neurons to work with.

Let's train the model:

• Press 'Train'. Again, this should take a while (like 30 seconds, depending on your CPU), be patient.

• Change Loss Function to 'Categorical Crossentropy', as is usual for Multi-class Classification problems.

• Change 'Show metric' to 'Accuracy'.

• Epochs: 30. This is a very big dataset, we don't need many epochs (cycles through the dataset).

• Batch size: 200. With 60,000 samples, updating node weights every 200 samples should be enough.

• In 'Show', select the first two options.

These are the training settings:

• Press 'Train'.

Training should take a couple minutes, depending on your CPU. These are the results:

Our model's accuracy is looking pretty good on the training dataset, at 94% accuracy. Let's test the model with the testing dataset, just to make sure.

• In 'Test File', select 'digits_data_test.txt' in the 'Data' folder.

• Press 'Test'. This should take a couple seconds.

• Click 'Browse' and let's name our output file 'digits_output'.

• Press 'Test model'.

This should also take some time. After it's over, you should have the 'digits_output.txt' file inside the 'Data' folder (or whichever folder you selected).

It's confirmed, the model is correctly identifying handwritten digits 93.4% of the time, that is, with an error rate of 6.6%. Here are some of the outputs of the model, compared to the original images:

The image above has the first 8 images in the testing dataset, and the model got them all correct! That should be expected given the measured accuracy of the model. In total, from the 10,000 samples, it only got 663 wrong. Let's now take a look at some examples of wrong classifications:

In the image above, we can definitely see why the model got them incorrect, as the digits are distorted and have weird shapes. However, some people do write like that, so in this state, this model can't be completely trusted for handwriting recognition. We would have to train it more, maybe with 100 epochs, or a different training set.

We've just created a model that correctly identifies handwritten digits about 94% of the time. That is very good for such a simple model.

However, there are other types of layers in Keras (besides Dense, the type Neurons uses) that are made to work with images, for example, the Convolutional layer (Conv2D). For a guide on how to use them in Python, check out this guide.