Not every classification problem is binary. We often have classification problems with more than two class labels. But if a perceptron only outputs a 0 or a 1, how can we use a perceptron when we have more than two classes? 

For a multi-class problem, we can have one perceptron per class label. Let’s say we are trying to learn to classify animals and our class labels are bird, reptile, mammal, fish, amphibian. We could have one perceptron that classifies the record as bird or not bird; then another perceptron that classifies the record as reptile or not reptile; and so on.  

The problem with this is that what if many perceptrons output a 1, indicating that the record is, say, a bird and a fish and a mammal? 

Well, instead of using a step function, which makes the output either 0 or 1, we can remove this step function and instead use the result of the summation as a ‘score’. The perceptron that outputs the highest score will be the class that we choose for this record. 

So, if the ‘bird’ perceptron outputs a 50 as the result of its summation, and the ‘fish’ perceptron outputs a 60, and the ‘mammal’ perceptron outputs a 10, then we would classify this record as a ‘fish’. 

Now how do we learn the perceptron weights in this multi-class, multi-perceptron scenario?

We still want to look at the error to determine whether we need to increase or decrease the weights, but now we need to do this for many perceptrons, instead of just one. 

First, we will check: Was the predicted classification correct? If yes, then we do not need to adjust any weights. If ‘fish’ came out highest, and the record is a fish, then all of the weights seem good, so don’t adjust anything.

But, if the predicted classification came out incorrect, then we want to make some weight adjustments. If the prediction was ‘fish’ but the actual label on this record is ‘bird’ then that means we need the ‘fish’ perceptron to output a lower value and we need the ‘bird’ perceptron to output a higher value. So we are going to update the weights on both of these perceptrons.

For the predicted-class perceptron (fish, in this example), we subtract the feature values from the weights to make the output go down:

wi = wi - xi

And for the actual-class perceptron (bird, in this example), we add the feature values to the weights to make the output go up:

wi = wi + xi 

Then we repeat this for every training record, and we repeat for some number of epochs.

Pause here and make sure you fully understand this! If you need help or have any questions, just turn your cups to red.

Representing multiple perceptrons in code

Again, pause here and make sure you fully understand this. If you need help or have any questions, turn your cups to red.

This list of 784 0’s and 1’s will be the input features for your perceptrons.  If each record/image has 784 features, how many weights will each perceptron have? And how many perceptrons will we need for this classification problem?

Take a look at the file perceptron_multiclass.py in the Exercises folder. This is where you will implement your digit classifier. 

You’ll notice that this file is set up just slightly differently than preceptron_binary.py in that the functions are inside of a class in the perceptron_multiclass.py file (there was no class in perceptron_binary.py, just the functions). Really the only thing that this affects, in terms of your coding, is that you now need to reference some of the variables using self.variable_name (as opposed to just variable_name). Note that the class variables that will need to be referenced via self.variable_name are: self.labels, self.epochs, and self.weights.

Other than that, the structure is the same as what you had before: You have two functions that you will need to implement. 

classify() will take in a record and use the perceptrons’ weights to return a predicted classification. 

train() is where your classifier will adjust all of the perceptrons’ weights as it learns from the training data. 

A few differences from the previous task to note:

• This function takes in the training data set, called train_data, 

• The labels have been separated into their own list now, called labels 

• The number of epochs to train for, can be accessed via self.epochs

Once again, print out the epoch and how many records were correctly classified during that epoch, so you can watch it learn.

epoch 0 .... 36.0% correct.

epoch 1 .... 78.0% correct.

epoch 2 .... 87.0% correct.

...

Run your code using the command:

python3 run_perceptron_multi.py

Once you have completed this task, turn your cups to red so that a member of the staff can check your code.