Max Mercer Math Museum - Simpson's Paradox

Simpson’s Paradox: When the Data Flips Upside Down

Imagine a hospital testing two new treatments: Treatment A (a targeted, intensive drug) and Treatment B (a general, lighter drug). You look at the data for patients with Mild symptoms: Treatment A cured 90%, while B only cured 85%. You look at patients with Severe symptoms: Treatment A cured 60%, while B only cured 50%.

The conclusion seems obvious: Treatment A is better for everyone.

But press the "Merge Patient Groups" button in the interactive above. When you combine the totals, the winner magically flips! Overall, Treatment B cures 78% of people, while Treatment A only cures 66%.

How is it mathematically possible for a drug to be better in every individual group, but worse overall?

This is Simpson's Paradox, a statistical illusion caused by unequal group sizes and a "lurking variable" (in this case, the severity of the disease).

Look closely at the split data: Because Treatment A was the "intensive" drug, doctors gave it to almost all of the Severe patients (40 patients), who are inherently much harder to cure. They only gave it to a few Mild patients (10 patients). Treatment B was given to the easy-to-cure Mild patients (40 patients).

Even though Treatment A is genuinely a better medicine, its overall success rate was dragged down because it took on all the toughest cases. This paradox proves why you should never trust a blind average without looking at the underlying context!

But what does this look like on a graph? Look below to see how this statistical trick can literally bend trendlines backward...

The Trendline Illusion

If you look closely at the chart above, a strange story unfolds.

We mapped software developers based on two factors: the Lines of Code they wrote (horizontal) and the Number of Bugs they introduced (vertical).

If you look only at the Junior Developers (blue), you'll see a positive upward slope: as they write more code, they make more mistakes. If you look only at the Senior Developers (orange), you see the exact same thing: more code equals more bugs.

Logically, if both groups show that more code leads to more bugs, then the overall trend across the entire company must be the same, right?

Click the "View Overall Trend" button.

The data suddenly snaps into a single, massive downward trend. When you view the company as a whole, it appears that writing more code actually results in fewer bugs.

This is the visual manifestation of Simpson’s Paradox. The illusion occurs because there are two distinct clusters resting at very different starting points. Senior developers write vastly more code than Juniors, but because they have more experience, their baseline number of bugs is much lower.

When you draw an aggregate trendline through the center, it connects the top-left (the high-bug Juniors) to the bottom-right (the low-bug Seniors), drawing a steep negative slope.

This is why context in data science is everything. If a researcher ignored the "experience level" variable and only plotted the overall trend, they would conclude that forcing people to type faster and write more code magically cures software bugs!

Continue scrolling to the final exhibit to see how this exact math error caused a famous real-world university scandal...

The Big Picture: Seeing Through the Data

You have just explored three very different worlds: a medical trial, a software development team, and a university admissions office. In each scenario, the top-level data told a compelling, logical story. And in each scenario, that story was a complete mathematical illusion.

Simpson's Paradox teaches us a fundamental lesson about living in an information-driven world: numbers do not speak for themselves.

Whenever data is simplified into broad averages, crucial context is often stripped away. As you have seen, lurking variables ... like the severity of a patient's disease, the experience level of a programmer, or the baseline competitiveness of a college major ... can completely invert the truth.

How can you protect yourself from falling for these statistical traps in your daily life?

Question the Averages

Whenever you are presented with a sweeping statistic, ask yourself if there are hidden subgroups or unequal sample sizes that might be skewing the results.

Look for the "Why"

Don't just blindly accept that a new medicine is better or a university is biased. Ask why the data looks that way and what underlying factors might be driving the numbers.

Demand Context

Real-world data is messy and nuanced. If a chart or a headline looks too perfectly simple, it is likely hiding a much more complicated reality beneath the surface.

The next time you see a sensational claim backed by "hard data," remember the token sorters, the shifting trendlines, and the admissions flows. Always look a little deeper, because the truth is usually hiding in the details!