Test data

Next scheduled ride: - (on hold till firmware update)

I'll organize the data in batches. A new batch starts with a fresh calibration ride, and all following training sessions before the next calibration ride will be included in this batch. To switch between rides, please click on the tab of the corresponding sheet at the bottom of the embedded spreadsheet.

Formatting of graphs should be consistent. Cadence is plotted on the right scale, and all power series are plotted on the left scale. If 'Difference' is >0, Stac Zero sends larger power values than Limits, and if 'Difference is <0, Stac Zero sends lower power values than limits. Note that the graphs contain a rather large amount of data and can take a second to load.

Since Limits is pedal-based and Stac Zero is post-drive train, Stac Zero could theoretically read 2-3% lower (or a bit more, depending on the state of the drive train). However, Stac Zero is total power, while Limits is left-only. Depending on my left/right balance (which I don't know), differences may occur. Since Stac Zero measures total power, I intend to do a left/right test soon - I have the idea that I should be able to estimate my left/right balance with some left-only/right-only intervals at different power levels.

I'm not the strongest cyclist. With 1.72m height, I'm at 67-70kg (depending on my shape), and my FTP is around 160w at the beginning of this test data.

Batch 1 (18/01/2017)

After installing the LIMITS and updating the firmware, I did a calibration ride, but I was interrupted twice. Also, I was running IpWatts in the foreground, and I've figured that this can cause the app to get stuck. When the calibration ride shows flat watts larger than 0 for a short period of time, IpWatts has lost connection to the power meter/power meters. Running IpWatts in the background fixes the problem and logging works just fine. Except for the calibration ride, I'll add a note for each of the rides.

Ride 1: 18/01/2017

Regarding the first real ride, I note a few things. Both power meters started in close proximity, but the lines separated when I moved in the first 15min/95w recovery block. Most notable, the 3min interval just before that block is the one with the best overlap of LIMITS and Stac Zero. Unlike all other intervals, this one I did standing with reduced cadence (70-75rpm). Afterwards, LIMITS switches from sending too low values to sending too high values. Note that Stac Zero power values have no drift: same effort, same power.

The big negative spikes of Limits at the beginning of some intervals seem to be related to me shifting. They systematically appear when my effort level changes. And while the Limits generally tends to overestimate power, it underestimates peak power. I have no explanation for overestimating, but could imagine peak power to be affected by some left/right balance issues.

Batch 2 (19/01/2017)

Since the first calibration was rather messy, I decided to give it another try and to do another 10min calibration ride. The same day, half an hour after completing the calibration ride, I participated to a 30km Zwift race (ZTR Thursday Training Race).

The re-calibration ride went okay. To me it looks like Limits and Stac now generally move in parallel. Unlike in the first batch, Limits stays consistently below Stac. Like last time, it underestimates peak power. Could be a software problem, or an issue of left/right imbalance. I only see that DC Rainmaker finds the same issue with peak power, also compared to other left-only power meters.

Ride 1: 19/01/2017

Then I re-calibrated for the zero offset and started the Zwift race, or, more precisely, the warm up to the Zwift race. A few minutes in, I noticed I had forgotten to open the window and quickly got off the bike and back on. That's the first gap a few minutes in. Afterwards, things continue very neatly. Note that I'm in low watt numbers, and that inaccuracies of the LIMITS in my rides usually appeared once I was a few minutes in.

Unfortunately, I run into a problem after about 11 minutes, 3 minutes before the race started. Apparently, IpWatts had lost contact to all sensors, and I couldn't get them back. Hence, the file is of little value, since it doesn't show the behavior of the Limits in a full intensity workout as I had intended. For what it's worth, it seems that this time there were some issues with the Stac Zero, while for the brief time of the ride, Limits and Stac aligned pretty well. I'm off the bike for the weekend, but curious how things are going to work out next week.

Ride 2: 23/01/2017 (FTP Test)

Having had four days off, and being one month into winter training, today was a good day for an update to my FTP. Aside of being satisfied with the outcome, that was also a good occasion for another LIMITS test. Results are now added in the Google sheet for batch 2. (There is a new work sheet you can open.) As usual, I quickly calibrated before starting the workout.

Both power meters read near perfectly similar. Really nothing to complain about during the warm-up phase. Differences only occur during the quick power burst sessions, then again during the 5-minute power interval, and finally during the 20min FTP test interval. In other words: Every time I actually put in real effort and don't just casually cycle along. It's a pattern that is not too dissimilar from what I have seen before. So, it could be that the strain gauge is not sensitive enough to higher wattage numbers (which might explain generally less accuracy that has been the testimony of other riders over at DC Rainmaker - riders which are most certainly stronger than me), or that it has troubles not with large absolute power, but with large abrupt changes in power, or that I have a strong left/right imbalance which only shows once I need to put in effort. Or all of that. Or something completely different.

Ride 3: 25-01-17

A note to start with: I have not shared this data with LIMITS support, because the workout data might be compromised. I had multiple issues throughout the session culminating in a complete breakdown of all sensors. Nonetheless, let's quickly go through it. I lost heart rate after about 8 minutes. After the ride, I've switched battery, but have not been able to get any signal from the heart rate sensor again. (But I can't exclude that the "new" batteries are instead old as well.) My immediate replacement had expired as well. Bought new batteries the next morning - that fixed it. Power readings continue as normal anyways, but LIMITS sends lower power than Stac Zero. Much lower power. I am speaking of -50w once I hit the first interval at about 15 minutes into the workout.

Unfortunately, it stays like this. And LIMITS shows very strange negative spikes for a second or two at the beginning of each intense interval, when I ramp up my power. It's a well-known story for me by now: LIMITS in firmware 2.1 struggles with high power, where high power is anything above ~100w. Since my FTP test yesterday adjusted my FTP from 157w to 180w, new workouts for me are going to have substantially more time with >100w. I expect to see much less flattering comparisons than before for that very reason.

However, today, things went nuts at the beginning of the fifth interval. LIMITS continues to send stable but wrong data to IpWatts, while Stac Zero's values are all over the place. For the sake of the workout, I continued, deducting the correct power from consistent cadence/gear combinations. That worked - until it didn't work anymore. I figured later that the spoke magnet had moved. Easy fix. Yet, what could have been a nice ride was kinda spoiled by all these sensor issues.

So, ignore the funky end of the workout, and take this whole ride generally with a grain of salt. Still. Overall, it confirms the emerging pattern: Indoors, LIMITS is limited to accuracy at low powers. There is one more ride planned for this batch, after which I will disassemble and reassemble LIMITS to see if that has any effect on accuracy. If it has not, I will stop testing with this firmware and wait for LIMITS support to send me or to publicly release a more recent firmware.

Ride 4: 26/01/2017 (Zwift race)

Yes, I managed not to have any technical issues today. So, this is beautiful data to look at. There's also something new to the story. Look at the initial part of the workout. There is some casual warming up, always around 100w. LIMITS and Stac Zero are perfectly aligned. At the end of the warm-up, I did three small power bursts, but I didn't shift for them. I wanted to see how LIMITS would react to abrupt adjustments in cadence. And look, it's spot on! Perfect! But...

As the race starts, I shift up four, five gears, since obviously I can't do the race with a cadence of 130rpm. LIMITS responds with one of its commonly sighted power drop outs (the negative spike). As of that moment, LIMITS gives power values that are too small. The gap is about 40-50w. A while later, a bit after the 3000s marker, the gap gets smaller, like 15-25w. This is how it stays towards the end. There, LIMITS does not capture the full power of the final sprint to the line (which I lost, by the way).

Take-away: perfect accuracy in the warm-up, until the first significant shift (i.e. more than gear in a short time frame). Afterwards the same picture as before, accompanied by some drop outs when I shift (but they don't always occur at every shift).

Batch 3 (28/01/2017)

By now, LIMITS has had a chance to demonstrate its behavior under various circumstances as they can appear in indoor training. It keeps underestimating power at higher power levels, so I wanted to see if the assembly procedure might have had an influence on its accuracy. I took the pedal off the LIMITS and the LIMITS off the crank arm, then put everything back on. Then I did a 15min calibration ride which I also used as a warm-up for another Zwift race I was doing this morning. I varied power output, both by varying cadence and gear. All looks like it used to. At the end of the re-calibration ride, I got off the bike to catch a power bar, then got back on and continued pedaling already while I stopped the log.

Ride 1: 28/01/2017 (Race)

Only when writing this up, I realized that I actually forgot the zero-offset between the re-calibration ride and the log of the actual ride. So, I'll add one more workout that I have planned for the evening, and we'll see how that goes. I'm not enough of a power meter expert, but from my understanding, the zero offset is meant to adjust for differences in conditions between the re-calibration ride and the next activity, and since they directly followed up on each other, the conditions were virtually identical. Still: I'm no expert, and I forgot it, so I cannot exclude that this might introduce some noise.

On to the actual ride. The first few minutes, I'm still in the warm-up phase. In the last minute of the warm-up, I gradually get up to power in order not to loose contact at the very beginning. Well, that worked, and I was able to keep contact to the race leaders all the way till the foot of Zwift's version of Box Hill. The small up-and-down in power until about 1400 seconds into the race is due to me trying maintaining my position in the group. I know that my cadence is rather low - I'm not a high cadence cyclist. The Box Hill itself has a more flat power output, because I was by myself, just maintaining a steady rhythm. On the downhill, I wanted to get my heart rate down, so I stopped pedaling as soon as Zwift had me at >59km/h. This is what looks rather messy around 2200s-2300s into the race. A sprint up a small climb and an acceleration towards the finish line, then cool-down.

Now, in terms of accuracy, it's the same story. LIMITS is rather accurate at low power levels. LIMITS consistently shows less power than Stac Zero, even where the difference is small. While the latter I would attribute to left/right imbalances, the power differences at higher power levels are simply too large for that explanation. My conclusion is that either the strain gauges are not sensitive enough or the algorithm does not work well enough. But I recognize that LIMITS and Stac Zero come close together again at the very end of the cool-down.

In comparison to previous rides, there are less dropouts this time, but I also shifted less, maybe. This looks fine to me anyways. I also find that it responds absolutely instantaneously once I change power levels. It just does not get right by how much I change it.

That said, I'll add one more workout from tonight because I forgot the zero-offset. It will be a series of short"hill attacks" (1min high power efforts) and low-power recovery intervals.

Ride 3: 28/01/2017 (Workout)

Now with zero-offset. Did it matter? Nope. LIMITS and Stac stay close together until I cross 100w. They stay rather close together through the first three high-cadence intervals, though the LIMITS responds more hectic, likely because it's a bit more sensitive to cadence changes. The gap between LIMITS and Stac Zero opens a bit once I do the 5min tempo interval prior to the hill simulations on medium cadence (85-90rpm). Once more, LIMITS fails on the high-power intervals, giving me about 50w less than Stac Zero. It also performs worse than usual in the initial recovery intervals between the hill simulations, but this gets better over time.

As I've said, I'm not an expert on power meter measurement, and specifically not on issues of left/right imbalance. Previously, I thought that the total power would be the sum of the power of both legs. Obviously, that's wrong thinking. Each leg needs to apply 250w over each leg's power-input-phase (for simplification: while the leg moves from the top to the bottom of the circle) to keep me going at 250w. That's likely also not completely true, but less wrong than my previous thinking. Consequently, there is no way that these 50w difference can be explained (only) with any imbalance issues.

But one should not rush to conclusions. Bear in mind that DC Rainmaker's trainer tests of the LIMITS showed very different issues: jumpiness of the power values, drift and underestimating power, and just understating power in really high-power sprints, but being spot-on for the rest. Note that he disassembled and re-assembled the LIMITS between the first two trainer tests and the last one.

Also, I'd like to point at one particular section of his test of the Garmin Vector power meter from 2013: "Pedals tightened to 25 foot-pounds (ft-lb). Too little or too much can result in inaccurate power. If you firmly tighten with a wrench, you will be fine. Just don’t under tighten (e.g. finger tighten) or put your full force into tightening the pedals. " - Given that LIMITS' construction is similar, I cannot exclude that something like this might be at play. LIMITS asks to tighten the power meter to 40nm. My torque wrench only goes up to 15nm. I used it and then gave it some - but it might not be tightened as much. In an e-mail to me two weeks ago, LIMITS support had said that things might take a bit longer to settle in and therefore recommend a slightly longer calibration ride (so I did 15min, not 10, and then there was the race effort). They did not have any experience in terms of how that would influence accuracy.

At this point, I have to accept that I currently will not get better power values out of LIMITS unless (1) I get hold of a more appropriate torque wrench (I'll try), or (2) LIMITS makes a firmware update available. That also means that adding more workouts has very little meaning at this point. If something changes, I'll leave a comment again on DC Rainmaker's site on the LIMITS and on Indiegogo. I'd also be happy to post power data from other LIMITS users that may or may not look similar.