Phase 2

Creating the Model:

I create a simple convolutional neural network model and train it with our training data. I used a neural network architecture similar to the one in TensorFlow MNIST tutorial. The neural network has 3 convolutional layers for feature extraction and 2 fully connected layers for regression. However, I need to change a few variables from the MNIST tutorial. I only have three labels compared to MNIST’s 10 digits, and the image sizes for our pictures are different.

Furthermore, since there are only about 2000 training images, I need to run the training 100 epochs to reach the accuracy I wanted. The trained model reached an accuracy of above 95% for training and 84% for validation.

What the program does:

First of all, I need to convert RGB images into Grayscale ones and transform them into a 1d Array of images in order to feed them into the neural network model for training. Then, training data is split into training set and validation set by a 80:20 split. Furthermore, separating the data set into batches, so that our model is optimized by Stochastic Gradient Descent method. And with the reading of every batch, all images are fit within the computer main RAM and GPU’s memory. The model accuracy become better as the training progresses. The model is validated via the validation data, the validation accuracy improves along with the training accuracy of the model. I also have to randomize the order of the training images to eliminate any label patterns in the data collections. By doing so, I increase the accuracy of the model if it were applied to a different track from the track used for data collection.

Considerations added for this Project:

Because we hosted the track in a setting close to windows, the lighting of the track would always be different, causing the lighting to dramatically change the resulting image.

In order to account for this, I add a batch-normalization layer to the neural network to normalize lighting, so that it minimizes the impact of this issue on the model. The neural network is rather deep with several convolutional layers to make our model quite accurate. Finally the neural network architecture is shown in this diagram

Furthermore, training the model on a different computer with a Nvidia GPU is preferred for fast training the model, since the Raspberry Pi does not have enough memory and is not powerful enough to train the model.

My Experience:

Creating the actual machine learning part of the project was the most interesting part of it. After the painfully meticulous process of collecting data, creating and training the model was not only challenging but also extremely fun. I used the MNIST code as a base, but soon ran into some pretty big problems. Because the track was placed where sunlight could change the lighting of it, I had to add an extra layer to normalize the lighting in the images; otherwise the car would find some correlation between lighting and direction which was not good. Furthermore, I found that the convolutional layers improves the accuracies. Initially, with the first few models I created, the robot completely failed. Most of these was due to the data collection process. For the first few batches, the images were focused incorrectly, allowing the robot to only see about 3 inches from where it stood, making it permanently go forward. Next and probably the most unusual discovery, changing the style to where the robot will turn as soon as possible instead of turning at the last moment when collecting data actually helped train the robot.