HOW we Approach problem solving

Why most analytics projects start in the wrong place — and what to do instead 

Most analytical projects start in the wrong place.

Someone decides they need a metrics dashboard. Or a win-probability model. Or a points projector. The tool gets chosen before the question gets fully understood, and the engagement proceeds from there — gathering data that fits the tool, building outputs the tool produces, and delivering results that answer the question the tool was designed to answer rather than the one the client actually had.

We've seen this pattern enough times that we've given it a name: method-first thinking. And it's the single most common reason analytical projects produce technically correct results that nobody acts on.

At ORRO, we do it differently. We start with the question.

What do you actually need to know?

This sounds obvious. It isn't.

When a client comes to us, they often arrive with a solution already in mind. They want a dashboard. They want a model that predicts. They want a value calculator. These are reasonable starting points — but they're answers to questions that haven't been fully asked yet.

Our first job is to reframe the conversation with curiosity.

What decision is being made? Who is making it? What would change their mind? These questions set the stage for the real work to begin. An analyst who skips them is building a bridge to a destination nobody has confirmed.

Let the data tell you what's there

Once we understand the question, we resist the urge to immediately impose structure on the data.

This is counterintuitive. Most organizations start by trusting the standings — points in the win column are the measure everyone already agrees on. That works well when a team's record and its underlying play are telling the same story. It fails when they diverge, because a team's position in the standings can hide exactly the signal you need.

We saw this directly while building a Cup-prediction model across a full slate of NHL teams. One club — Buffalo — was losing early in the season, and a points-based projection called for a rough year. But when we pulled shot differential, the picture flipped. Shot differential (SD) is simply shots for minus shots against: a team generating more shot attempts than it allows, game after game, even while losing on the scoreboard.

Buffalo's SD was solid even while its record wasn't. That divergence — bad points, decent SD — was the tell that a pure points-based projection missed entirely. It turned out to be exactly the kind of signal that matters: not what happened, but what the underlying play said was likely to keep happening.

The lesson: exploration before classification. Let the data show you what's there before you decide what to measure.

Combine methods that don't usually travel together

None of our major projects has been solved by a single method.

The Hurricanes shot-differential work combined web-scraped possession data, points-based projection, and a separate shot-differential-based projection — run side by side because each one caught something the other missed, then compared to see which was more predictive in which situations.

The athlete-longevity research combined survival modeling, regularized regression, and landmark regression — a technique that predicts whether a player who has already reached one milestone (say, surviving to season three) will reach a later one (season five), rather than trying to model an entire career length from a player's first game.

The 2026-27 points projector combined multi-year trajectory modeling with a four-stage career framework, treating a player's early usage, draft position, and physical profile as separate analytical problems rather than one blended one.

In each case, the method combination wasn't chosen because it was fashionable or because we had a favorite tool. It was chosen because the question demanded it. That's the only legitimate reason to choose a method: because it's the right one for what you're trying to find out.

The null result is a finding too

One of the things that distinguishes rigorous analytical work from confirmatory storytelling is the willingness to report what the data didn't find.

In our athlete-longevity research, we built an extensive set of variables designed to capture the psychological and social dimensions of a player's career — team stability, roster continuity, how long a player had been with the same teammates. The hypothesis was that the human environment around a player would add meaningful predictive power beyond the observable physical facts.

It didn't.

After controlling for early-career usage, age at NHL debut, and physical characteristics, the psychological and social variables added almost nothing to the model. Four observable variables did the vast majority of the predictive work. Everything else was marginal.

We reported that clearly. Not as a failure — as a finding. The data was telling us something important: that by the time a player reaches the NHL, how heavily they're being used in their early seasons is more informative about their eventual career length than anything we could currently construct to capture the human side of their experience.

An analytical partner who buries null results, or spins them into something more palatable, is not actually serving their client. They're serving their own desire to have found something. 

Translation is half the work

The most technically sophisticated analysis in the world accomplishes nothing if the people who need to act on it can't understand or trust what it's saying.

This is not a soft skill. It's an analytical skill — and one that gets far less attention than it deserves.

Our shot-differential work on the Carolina Hurricanes is a good example of why. Going into last season, a raw stat like "+8.24 SD per game" doesn't mean much unless someone translates it: this team is out-shooting opponents by eight to nine shots a night, on average, and that kind of gap almost always reflects a genuine structural advantage — not puck luck. We said, in October, that this was a team built differently.

By June, Carolina had backed it up, winning the Stanley Cup and closing out both the Eastern Conference Final and the Final itself by dominating puck possession exactly the way the numbers predicted. The finding only matters when it is translated into language people understand — not because it was correct in hindsight.

Every engagement we undertake ends with a communication challenge. Who is the audience? What do they already know? What level of technical detail serves the decision they need to make, and what level obscures it? We think about these questions as carefully as we think about model selection. A finding that gets nodded at in a meeting and then filed away has failed, regardless of how correct it is.

The question that drives everything

We are drawn to problems that don't have obvious solutions. Problems where the right method isn't clear at the outset. Problems where someone has tried the standard approach — the standings, the scouting report, the box score — and found it insufficient.

That curiosity is what connects a shot-differential model built on a Stanley Cup contender to a survival-modeling study of NHL career length to a multi-year points projector. Those problems look different on the surface. Underneath, they share the same structure: a question that matters, data that is harder to analyze than expected, and people who need to understand and trust the answer before they can act on it.

Start with the question. Everything else follows.

© 2026 ORRO AI Genius, LLC. All rights reserved.