Updates
Weekly Progress Reports
Week 7
We initially proposed the idea of using an STM32 microcontroller to perform object recognition tasks. Unfortunately, we realized that the STM32L4 Discovery Board would not have sufficient resources to perform the computation we needed.
Luckily, Professor Isukapalli gave us the option of using the MAX78000EV kit that was developed to build ML application solutions. After some research on the board, we decided to try out the new board and came up with a new proposal: using an ML voice recognition model to accept voice commands for the purpose of controlling a smart lightbulb.
Week 8
This week, we purchased the MAX78000 EV kit and the LIFX A19 smart bulb. We were able to enable remote control of the bulb via a Python script. The video to the right shows the process of turning the bulb on and off.
After some trial and error with a new development environment (Maxim Eclipse), we were able to successfully install and run an ML voice recognition model on the MAX78000 EV board. The model is able to recognize the following keywords: ['up', 'down', 'left', 'right', 'stop', 'go', 'yes', 'no', 'on', 'off', 'one', 'two', 'three', 'four', 'five', 'six', 'seven', 'eight', 'nine', 'zero'] and display the probability of the most possible words on the TFT screen. The video on the right demonstrates the model in action.
We are currently considering adding a facial recognition function.
video-1614063160.mp4sd1614062125_2.mp4 Week 9
This week, we were able to establish a connection from a microcontroller to the HC-05 Bluetooth Module to a Python control script to an LIFX lightbulb. For this prototype, we used the STM32 Discovery board in place of the MAX78000 EV board as a proof-of-concept, using the onboard joystick control buttons of the Discovery board. The video to the right shows our prototype in action, and although it may be a bit difficult to see due to the video quality, the LIFX bulb's brightness level is adjusted by the Discovery board's joystick buttons.
PXL_20210306_045909301.mp4 Week 10
This week, we spent a significant amount of time attempting to interface the HC-05 Bluetooth module with the MAX78000 board. Unfortunately, although we were able to establish a connection, the data sent over the connection was garbled and uninterpretable. With guidance from Professor Isukapalli, we decided to opt for a serial UART connection over USB.
We also decided to introduce a user authentication phase using facial recognition. After selecting and verifying a pretrained facial recognition CNN model, we attempted to get the the model to run simultaneously with our voice recognition model on the MAX78000 board. Unfortunately, despite our best efforts, the board crashed every time we attempted to initialize the two models together.
Finals Week
Despite our setbacks during Week 10, we still wanted to create an authentication phase using a facial recognition model. We decided to change the architecture of our project so that everything would be controlled by an automated central Python script. This script loads the facial recognition model onto the MAX78000 board using OpenOCD and GDB and performs user authentication. Upon success, the automated script then loads our voice recognition model and adjust the LIFX bulb according to voice commands. After some significant setup work, we were able to implement this. We also customized the TFT display so that it displayed relevant info during the authentication phase.
The final product is a system that meets our goal of performing user authentication using facial recognition and smart bulb brightness adjustments using voice recognition.