COM S / HCI 575 Final Project: Smasheo

For my final project in COM S / HCI 575, a friend and I created an application that tracks characteristics of a game of Super Smash Brothers Ultimate. It tracks character position, what attacks are being carried out, although we only had time to get a subset of attacks being recognized for one character, how many lives each character has remaining, and airtime tracking. Furthermore, it detects how much damage each player has sustained and generates a heat map that shows where a player spent the most amount of time. It's original goal was to give feedback to a player to help them improve, but we still need to incorporate more attacks in our detection in order to give meaningful feedback. Over all, we were very successful with all of our tracking, with the exception of shield tracking.

This project helped me understand signal processing, and helped me learn new computer vision techniques. In the project, I was responsible for player location tracking, attack recognition, player stock detection, and shield recognition. This included all audio processing, since attacks were detected with both audio and visual ques. All of the specific details of the project are laid out in our final paper below.