Experiment and Results
Experiment and Results
Embedded Files
Experiment
Experiment
We performed three trials of the experiment. In each, the subject ran two laps around the track at the Burns Track Complex at Claremont McKenna College. For the data collection, two phones running the phyphox app were required to collect accelerometer and gyroscope data, respectively. These phones were strapped together and securely held to the subject's chest for the duration of the run. Throughout the run, the subject's GPS watch continuously collected data as well. Every trial used the same path: starting at the (0,0) mark and running two laps, always in the inside lane.
To evaluate the performance of the state estimator, we can compare the output of the EKF to the GPS data collected. As the path is of known shape, we can compare the performance of the GPS data to the performance of our state estimation of the tracked run. To do this, we compared the paths against the theoretical model quantitatively using the RMS error over time and qualitatively by visually comparing the estimated path with the theoretical path, which is known.
To calculate this RMS error associated with each method, we can use Equation 1 where n is the number of data points and e is as defined in Equation 2. Equation 2 details how the error is defined by comparing the path estimation at a given time step to the true path (the known shape of the track).
Using the calculated RMS error for both the GPS data and our state estimated path, we can quantify how successful the project’s state estimator is compared to the current standard for runners.
Results
Results
Presented below are figures of the data that was collected during one of the trials of the experiment. Specifically, figures of the acceleration, heading, and GPS are pictured.
Shown first is the acceleration. As seen in the untrimmed version in the middle, the data requires a lot of preprocessing before it can be used in the filter. Specifically, this figure shows the large amount of acceleration that the phone experienced throughout the trial, which contrasts the "beginning" and the "end" of the trial when there is very little acceleration (when the runner was at a stop). Additionally, when the phone was taken out of its holder right before the "end" button was pressed on the data collection, the acceleration spikes again. This noise corresponds to experiment set-up, and not experimental data. As such, we make sure to trim this data before using it. Additionally, there is a moving average filter applied to the data to smooth out the trends; the figure below on the left shows this trimmed and filtered data. As expected, the acceleration is cyclic and corresponds to four curves: two for each lap around the track. It is interesting to note that there are cycles present in the data that correspond to footsteps, as shown in the below figure to the right. This affects the final result of the filter, as it can only be as accurate as the data it uses.
Filtered accelerometer data.
Filtered and zoomed in accelerometer data showing oscillation from footsteps.
Next is the heading data collected by the gyroscope. Again, this collected data looks as expected. Two full laps around the track are shown in this cyclic data. Additionally, it can be observed that the heading is wrapped to be between -π and π. The runner starts with a heading of zero, then reaches the first curve in which the heading steadily increases, until it is flat again going the opposite direction, and finally hits the second curve where the heading climbs back to zero. Something of note in the data shown here is that while we expect the heading from 100 to 125 seconds to be a flat line at zero, it sits above this expectation at ~0.75 radians. Additionally, the heading does not generally stay at a constant value on the straights; this is a result of the noise in the system.
The next figure shows the GPS data that was collected during the trial. It can be seen that the GPS data is more accurate for the straight parts of the track than the curves. This is the data that we seek to improve with our state estimation project.
We performed our EKF on the data presented above, which results in estimated paths as shown in the two figures below. The left figure shows the result of our unique EKF filter with the theoretical curve correction. The right figure shows the result of our standard EKF filter. As seen, our unique EKF improves the GPS data as the estimated path is much closer to the theoretical path (especially on the curves) than the standard EKF gives. The figure on the right performs approximately as well as the GPS data shown above. The figure on the left, however, provides much more accurate results.
Estimated path using modified EKF (with curve correction). On the curves, the estimated path hugs the theoretical curve model. The lines in the middle of the track, especially toward the right of the image, are due to noise in the heading data that determines when the loop is exited.
Estimated path without curve correction. The runner path is farther from the theoretical track path plotted in orange.
There are seemingly some data gaps in the estimated path. This is likely due to the algorithm switching between the curve correction and the regular correction on the straights vs. the curves. In future iterations, this could be avoided by (a) collecting better heading data (or correcting/filtering it to be better) so that we can use heading alone to determine when we are on a curve or (b) adding conditions to use correction on the curves based on x and y position as well. However, the team hypothesizes that this is largely due to noisy gyroscope data, especially because it did not change significantly when changing the heading requirements and it occurs exclusively in areas where the algorithm switches from straights to curves. An algorithm more robust to this change could fill in these gaps, but generally the estimated path is better than the plain EKF, and interpolation could be used accurately to fill in these data points.
Shown below are the performance plots using RMS error over time as an quantitative measure of performance (using Equations 1 and 2 as presented above). While the error is nonzero in both cases, the modified EKF results shown at left are more promising, as the RMS error is reduced as compared to the results on the right for the standard EKF without curve correction. For the EKF with curve correction, the RMS error drops on the curves, demonstrating the method's success at matching the heading to actual locations on the track.
RMS error for the EKF with curve correction.
RMS error for the EKF without curve correction.
We can compare these RMS errors to the RMS error associated with the GPS path itself. As seen in the figure below, the RMS error for the GPS path is similar to that of the EKF without curve correction. However, the EKF with curve correction has smaller RMS values than the GPS. Therefore, the EKF with curve correction implementation used in this project was successful, as it tracks the runner's location more accurately than GPS paths (that apps like Strava use).
Page updated
Report abuse