The Fatal Flaw of Using Data to Make Big Decisions

Go with your gut or analyze the data? The answer to making smart business decisions may surprise you.

Share on
BY Scott Painter - 08 Feb 2018

The Fatal Flaw of Using Data to Make Big Decisions

PHOTO CREDIT: Getty Images

In my last column, I outlined how a near-fatal level of optimism is the most important qualifying trait for any entrepreneur. But even a startup founder with eternal confidence is likely to be sobered by what's sitting across from them on the see-saw of their business life: a giant gorilla named "risk." And the higher that entrepreneur's optimism, the heavier the beast on the other side is likely to be.

Mind you, not all risk is bad. Taking healthy risks is how you grow your business, and managing risk properly will be a critical part of every strategic decision you make along the way. But the same upside-focused thinking that can get a successful entrepreneur through the tough spots can also bring with it a penchant for taking on too much downside.

Protecting against this comes down to a simple concept we've all known since childhood: understanding the likelihood of a given outcome, and the benefits and drawbacks that might come from it. Sometimes this assessment can be fairly straightforward. If you're standing on a seaside cliff and think you can jump far enough out to improbably (but impressively!) miss the 20-foot outcropping of rocks below, you're dabbling in unnecessary risk. You don't have to conduct empirical testing. You don't need a focus group. JUST. DON'T. DO. IT.

Unfortunately, the risk inherent in most business decisions is rarely this obvious, and requires an evaluation using one of two methods: evidence-based decision-making vs. the good, old-fashioned option of going with your gut. Unsurprisingly--given the bad business decisions made every day--neither is a perfect solution.

For all of the reliance placed on AI and big data in the digital economy, evidence-based risk analysis is actually incredibly dangerous on its own--for several reasons.

First, any test can be skewed. You can have a poor test of a good idea. You can also do something to test a really bad idea that convinces you it's a truly great one. Throw a dozen executives, lawyers and consultants into this mix--each with their own idea of where things should land, and a finger or two to put on the scales--and a solution can often end up even more elusive than it was in the beginning.

Of course, one way of overcoming this is through sheer volume: run a risk model a couple of dozen times knowing you're not going to get it right for a while, then simply do what the numbers show you. Unfortunately, this laboriously swampy approach to risk can lead to a quant-based quagmire, killing both momentum and spirits alike. Performing complicated math to weigh any number of paths every time your company needs to take a step forward will paralyze progress and disenchant your most talented people. And when they realize their every insight and idea is being lab-tested, they will lose faith in your leadership--and they will leave.

On the other end of the risk-assessment spectrum, there's the aforementioned John Wayne method: making a gut call and sticking with it. In business, this act-first approach to risk management has some high-profile adherents. Steve Jobs was famous for his disdain of focus groups, while Henry Ford once said of making the first mass-market cars: "If I had asked people what they wanted, they would have said faster horses."

As much as I love that quote, the problem with this attitude as a pure approach to risk management is clear: not everyone is Steve Jobs or Henry Ford. So a good many leaders simply resort to the numbers-based method, or--just as likely--end up making truly terrible gut-call decisions.

Given all this, then, what's the best solution for an entrepreneur forced to make a risk-filled decision in a fast-moving business environment? Actually, it's a carefully measured combination of both approaches, consisting of an incredibly informed gut call put through a tightly defined analytic examination.

Apart, both exercises are deeply flawed for the reasons stated above. Together, though, they are quite literally the yin and yang of a ridiculously good risk analysis. And the steps to mixing this super-powered hybrid solution are actually quite simple.

Start with your gut call--an incredibly strong, almost binary thesis around how to pursue a given decision. It should be a singular, unwavering point of view that you have zeroed in on as a likely solution to the problem at hand.

Then run a comprehensive, but definitively scoped round of risk analysis on this single hypothesis, using a simple piece of criteria to define its success: whether it will make your product better. If it does--with all the structural and cultural implications this determination includes--pursue it. If it does not, drop it and move on.

If this process sounds a tad unilateral and terse, well, it is. And that's kind of the point. With the winds of the marketplace constantly in its face, your startup can't afford to get bogged down in time-sucking, data-driven game-planning for the countless strategic junctures you'll cross during your journey. In such instances, leaders need to rely on the same instincts and knowledge that got their company to the given decision point, and overlay their call with only enough statistical analysis to gain a win/lose determination, which should be unquestionably honored.

You're almost never going to land on how something will work best by empirical testing alone. The risk-management process doesn't create delightful outcomes or incredible solutions. That requires visionary thinking--the kind you should lean into if you're doing your job as a CEO.

So be bold in the face of risk. Be unapologetic about your instincts. But be ready to test them out with some of your really, really smart people first.