Technical Info

HOW WE DID WHAT WE DID:

To collect data for performance analysis, performers users record themselves performing one particular trick for at least 10 seconds while using our product to form a calibration dataset. Then, they record themselves performing their performance routine to form a performance dataset.

After this data is collected, it is processed using the Discrete Fourier Transform to highlight the frequencies present in each of the tricks. For the calibration dataset, we cluster the data and reduce its dimensionality with eigenvectors, which allows us to characterize each of the tricks. The performance dataset is divided into into many subsections of fixed length, and each subset of data is characterized and compared to the calibration dataset to identify the trick performed in that time interval.

Algorithms:

Motion Model Dynamics:

By observation, we noticed that smaller circles would result in a higher normal acceleration and larger circles would result in a . By taking a closer look at the dynamics of our motion model, we see that this is consistent with what we expect.

Due to the fact that the poi was constantly being spun along a circular trajectory, we can summarize the centrifugal acceleration (a.k.a the Y-accelerometer readings) of the poi with the equation

a = (m*v²)/r

where m = mass, v = linear velocity, r = radius

Since the mass is constant and the linear velocity was fairly constant throughout the trick, it is apparent that the centrifugal acceleration is inversely proportional to the radius the poi transverses.

This relationship is what aided us in characterizing different tricks. For example in Half Chasing the Sun, the poi would traverse in one large circle, followed by a smaller circle. When the poi was in the larger circle the radius was greater, and according to this relationship it would experience a lower normal acceleration. The inverse is true for the smaller circle. Since it has a smaller radius the normal acceleration is higher.

Frequency Analysis:

Both the calibration and performance data is collected as accelerometer values in the time domain. However, by observation, it is much more difficult to characterize data containing periodic motions (like the motions of poi!) in the time domain than it is to characterize this data in the frequency domain.

Based on this observation, we thought it'd be a good idea to use the Discrete Fourier Transform to map the data from the time domain into the frequency domain before using other methods to characterize the data.

The matrix that describes this transformation can be written as:

When this transformation matrix is multiplied by the time domain data, the result is this data mapped in the frequency domain.

Filtering:

To filter our dataset, we used two filters - a moving average filter and a notch filter.

The moving average filter was applied to the raw accelerometer data. The raw data had a decent amount of noise in it (in part due to small movements of the poi while the performer was holding them and in part due to the sensitivity of the sensor itself), so to filter this out we used a 7 point moving average filter on the data. By averaging out consecutive data points, the effects of these minute changes were minimized, reducing the miscellaneous high frequencies that were previously present in the data.

The notch filter was applied to the data after it was already transformed from the time to frequency domain. Specifically, we used a notch filter with a q value of 0.96 centered at 0. The peaks at 0 occur as an artifact of the DFT, and are related to the average value of the transformed data. Since the poi was almost always spinning in a circular motion, there was always a significant positive Y-accelerometer value. As a result, the mean of our data was almost always a large value, which caused a large spike to be present in most of the DFT results of the raw data. This peak would throw off our frequency analysis, so we decided to filter it so that we can focus on the characteristic frequencies that resulted from the dynamics of each of the trick.

Characterization with Eigenvectors:

To characterize each trick with eigenvectors, we need to have a training and testing dataset. Our training dataset was a matrix with the filtered Y-accelerometer data in the frequency domain for each trick as separate columns. For initial testing, our testing dataset was in the same format, but with different data. For identifying tricks within a performance, our testing dataset had the filtered Y-accelerometer data for a given time interval in the frequency domain as columns, and each time interval had its own column with frequency data.

First, we mean-centered the data then used singular value decomposition on the result to identify the principal eigenvectors. Eigenvectors point along the direction of greatest variance between the data and are sorted in order based on the relative variance - the first eigenvector points along the direction of greatest variance, the second points along the direction of second greatest variance, etc. When more eigenvectors are included, more "directions" of variance in the dataset are considered when characterizing each trick. These eigenvectors form the basis of an n-th dimensional space where the data is spaced out to increase variance between the different datasets.

Singular value decomposition also results in a transformation matrix that maps a vector from its original form to its form within this new space defined by eigenvectors. When this transformation matrix is applied to a vector, the result is a series of coefficients that can be used to characterize that vector within the eigenvector space. In summary, each trick can be characterized as a list of coefficients representing the linear combination of these eigenvectors required to form the trick. The minimum RMS error between coefficients from the test and training dataset yields the "most similar trick," allowing us to distinguish between the different tricks performed.

RESULTS:

We were able to get 87% accuracy on identifying individual tricks when each performer provided their own calibration dataset (we found that different performers performed the same trick differently enough that the same calibration dataset couldn't be used). However, this accuracy dropped to only ~10% accuracy identifying tricks within a performance. This could be attributed to the fact that our identification method required a lot of data to characterize a trick accurately, but dividing the performance data into larger time intervals caused multiple tricks to be present within the same interval, confusing our identification algorithm.