Why testing business ideas is a powerful way of transforming uncertainty into knowledge

8 minute read

Wonam on a beach wearing a blindfold
cameraPhoto by Oscar Keys on Unsplash

TL,DR;

Innovating means making something new. Making something new means uncertainty. The best way to deal with uncertainty is to get it out into the open and address it.

Begin by identifying the assumptions that are the foundation of your business idea/model. Write them down. Decide how much evidence you already have, if any, to validate those assumptions. Decide which assumptions are critical to the success of your business.

Collect together all of the critical assumptions for which you have little or no evidence and test them now. How do you test them? By turning them into hypotheses - proposed explanations for a set of circumstances that can be tested.

Then test, test, test. Oh, and learn from what you find!

I’m pink, therefore I’m spam

I recently worked with a start-up that is looking to shake up the world of short-term renting by filling existing but currently empty rental properties at the last minute by lowering prices (compared to the standard rate). Their premise is that a low price is better than no price.

Their business model assumes that X% of short-term rental properties remained empty at any one time, a figure easily determined by looking at the statistics coming from the competing short-term property rental portal they hope to take advantage of. Their intention was to increase utilisation of these empty properties by Y%, representing an opportunity valued at £Zm.

This is not an unusual approach in the world of business, especially start-ups. In fact, it’s one I see all the time:

The worldwide market for widgets is estimated to be eleventy gazillion pounds. We’re aiming to capture 13.57% of that market with our patented widget wrangler technology which represents an annual revenue of two squillion pounds. Please invest in us

This kind of statement is OK if you have properly researched it and have figures and strategies to back up your argument. The problem comes when start-up founders assume that an opportunity exists simply because a market exists. If that market is large the temptation is to assume that you can capture some portion of it. Many properties lie empty for a reasonable amount of time so we should be able to fill them, right?

The answer is maybe. It’s possible that you have found an opportunity, but without further analysis any investor worth their salt will see straight through your figures and your business model.

When I asked the short-term rental start-up how they calculated that they could increase utilisation of empty properties by Y% their answer was that they looked at the total number of days booked vs total number of days not booked and assumed that they could simply reduce the number of unbooked days by a reasonable amount.

I then asked them whether they had accounted for possible environmental reasons for the variations in demand - for example, had they considered that the reason a property remains empty on a wet Tuesday in January is because no-one wants to book a property at that time? They had not. Note that by ‘environmental’ I’m referring to the problem environment, i.e. the context the potential customer operates in, which is beyond the control of any start-up.

Ask yourself this: If this company had come to you for investment and you found out that they had not considered reasons beyond their control that might cause the variations in demand that they hope to exploit, would you give them any money? I know what my answer is: A firm ‘No’.

Cognitive Bias

Bias in decision making impacts our lives every day. We all have our own biases informed by our life experiences, education, culture and social interactions. We most often associate bias with negative impulses such as prejudice, but there is a category of biases linked by the umberella term “Cognitive Bias” which have come about through evolution and which have helped the human race survive. We often refer to this kind of bias as gut feel or intuition.

There are a startling number of cognitive biases that have been identified and what they mostly have in common is that they are great for making decisions when it comes to surviving and evolving as a species but are generally unsound when attempting to make informed business decisions.

For example, in the case of the start-up I described above, it could be argued that they are suffering from selective perception bias - their entire business model rests on the ability to fill un-filled properties, so there simply must be a way of filling them. To consider any other outcome is just too painful!

It doesn’t matter how we categorise the bias that is preventing deeper investigation of the problem at hand. What matters is that a gut feel on the part of the founders combined with a generic (and incomplete) set of data has resulted in a high risk assumption.

We all suffer from cognitive bias whether we are aware of it or not. Start-ups operate in a world of extreme uncertainty as it is, without the complication of cognitive bias making things more difficult. This is just one of the reasons why experimentation is so important to start-ups. Experiments help us gather evidence and make decisions. If our business model is not based on evidence, what is it based on?

How do we decide what experiments to do? We start by examining our assumptions.

Making an Ass of U and Me

The significance of an assumption depends largely upon the following factors:

  1. The negative impact if it were to be invalid
  2. How soon that negative impact would be felt if it were to be invalid
  3. The likelihood of it’s being invalid

There is a saying:

Never assume - it makes an ass of you and me

Innovation is realm of the new. In order to innovate we must, by definition, make assumptions. Making assumptions is not inherently bad. What is bad is not recognising that you have made and assumption and/or making no attempt to validate it.

Once we know what our assumptions are we can focus on those assumptions which are most critical to our business. Beliefs that must be true in order for a start-up to succeed are called leap-of-faith assumptions.

I believe you have a greater chance of succeeding if you find the fastest possible path to realising your vision. I’m suggesting that the the first, best step on this path is to identify and test your assumptions as quickly and cheaply as possible.

You do this by turning your assumptions into hypotheses. A hypothesis is a proposed explanation for a set of circumstances that can be tested. Once you have your hypotheses, the next thing to do is to test them and learn from the results. In Lean Start-up terms, this is called the Build-Measure-Learn cycle.

I don’t like this particular terminology, because I believe that words are important. Calling it the Build-Measure-Learn cycle puts the emphasis on building, rather than on testing which is where I think the emphasis should lie. For that reason, I like to call it the Experiment-Learn-Pivot cycle instead.

Whatever you want to call it, just do it. Start with your leap-of-faith assumptions.

Converting assumptions into hypotheses begins with breaking them down as far as they can go, until you can go no further (see this blog post for some suggestions about how to do this). You’ll know when you’ve gone far enough if your resulting hypotheses are:

  • Testable - They can be shown to be true of false based on evidence
  • Discrete - They each describe only one distinct, testable thing to investigate
  • Precise - You know what success looks like (and you define this before you run the experiment to test the hypothesis)

Assumptions can be both explicit (you are aware of them and recognise them as being assumptions) and implicit (they are so much a part of your thought processes and understanding of a situation that you don’t see them as assumptions at all).

One of the defining characteristics of cognitive bias is that the assumptions that result from it are overwhelmingly implicit. This is one of the benefits of running workshops with an independent third party: because this third party doesn’t share your way of thinking or familiarity with a situation they are better able to recognise and identify your implicit assumptions than you are.

Ask the right questions

There are 4 main reasons why business models fail:

  • You are trying to solve an irrelevant customer job - e.g. Segway.
  • Your business model is flawed - it costs more to acquire customers than you make from them (e.g. Kodak)
  • External threats are too great - e.g. there are cheaper competitors, the product/service is not environmentally friendly, there is a stockmarket crash.
  • Poor execution - e.g. the wrong team with the wrong skills, poor leadership. The Apple Newton is a good example of this.

When listing your assumptions, try to put them into one of these four categories. Ask yourself which of these potential issues your assumptions address. If you have few or no assumptions in any category ask yourself why that is. Is it because you have already examined these assumptions by forming associated hypotheses, experimenting, gathering evidence, gaining insight and taking action? Or is it because you haven’t acknowledged the assumptions you are making?

The take-away

Don’t get me wrong. I’m not saying that the short-term rental start-up I described above is wrong when they predict a return of Y%. I’m simply saying that I don’t know whether this return is remotely realistic and neither do they.

This is such a basic leap-of-faith assumption that it needs to be tested inside out because the consequences of it’s being wrong are disastrous for them with their current model.

The great thing about experiments is that they test your thinking. There is no such thing as a wrong outcome from an experiment. If it proves to be true then you can continue with confidence. If it turns out to be false then the sooner you know this the sooner you can adjust and succeed in different ways.

Leave a comment