Implementation

We have chosen to use a hardware implementation so that we can recreate the exact usage of the solution by the average consumer. The idea is to have a microcontroller that can make real-time predictions of objects and inform the user of which trash can to place the object in. Ideally the microcontroller would be attached to the wall with a camera mount that would face downwards towards the ground farther from the wall which would allow users to place an object in front of the camera and press the push button next to the camera. This would then predict the object very quickly and use a servo motor to raise the lid of the trash can. Due to the time restrictions we decided to focus on creating a hardware proof-of-concept implementation prior to creating a mount and servo motor powered trash cans.

We chose to utilize the Nvidia Jetson Nano for our microcontroller given the small footprint and ability to run neural networks for tasks like ours. Paired with the GPIO pins for buttons, motors and LED's, we can use it to make quick real-time predictions while still operating headlessly with its local memory.

After completing the project we were able to create a functioning push button and also make predictions using an attached webcam. In order to inform the user we simply labelled the video stream frames and it would be possible to create the entire solution if we had a mini display and camera mount. We trained the model on our local system and just made predictions on the model but making continuous predictions seemed to cause our frame rate to drop and so we determined that it would be better to make use of the push button for predictions which could improve the camera footage and also save battery by decreasing the number of computations done per frame.

A few key challenges we ran into:

• In theory, ResNet50 should have yield at least a comparable result in reference to our other two models. However, the accuracy differential was much larger than expected towards the negative end. Due to how we approached our models sequentially, we did not have as much time to troubleshoot and tweak the weights and hyperparameters.

• We found our model to be highly light dependent as well with varying degrees of accuracy depending on the exposure of our image.

• Package interoperability on virtual machines as well as the Jetson Nano.

For future improvements on the hardware side, stepper motors could be introduced such that the corresponding bin chosen by the model would open. This would allow for the user to simply place the object under the camera and then proceed to place it into whichever can is open. Our current implementation is non-permanent and relies on the user to hold the camera and the object so that we can pass frame through the model. As such, for more realistic prototypes going forward, a mount for the camera to be held uniformly above the bins could be 3D printed for easier classification. Addressing our interim solution for frame rate drops using the push button, we could also aim to solve this in a hands free way similar to other garbage cans that use sensors to detect that someone is trying to use the garbage can.

As for improvements on the model itself, we saw good performance for both VGG-16 and DenseNet201 against the test dataset but a differential in real time performance. Going forward, we could tweak our dataset to include varying angles as well as modifying our hyperparameters. This way, we could address variations in how users present objects to the system. In addition to this, readdressing our hyperparameters and fine tuning for our ResNet50 implementation could yield results better than our current state for VGG-16 and DenseNet201.