When you don’t know, run experiments

Image by PIRO4D from Pixabay

The effects of digitalization and other technological shifts cause companies to realize they need to change. This often leads to significant discussions in the organization as there typically are several alternatives being considered by different people. These might include topics such as business models, product implications, partnerships with suppliers and technology providers. Agreeing on the right way forward is difficult as it often is quite challenging to accurately predict the implications of changes to the way things are working today.

One of the biggest dangers organizations encounter in this context is to end up in analysis paralysis. For every change that anyone in the organization might propose, one or more doom scenarios are presented by others. The consequence is a fundamental deadlock where any change is immediately torpedoed by a fear-driven organization where the apocaholics always end up on top.

Another typical pattern many organizations experience is when a team is assembled and locked up in a room, with the expectation that it will come out with a proposal on how to proceed that’s entirely based on the opinions of the people in that team. The challenge is that, in most cases, the opinions of the team members are based on the current state of the business and the historical events leading up to today. Although many claim that history doesn’t repeat itself but it most certainly rhymes, it seems that the changes many companies are faced with cannot be predicted by extrapolating from the past. Rather, what’s required is a belief in a future that hasn’t yet arrived. And in order to be ready for that future, the company needs to jump into the dark.

The challenge with complex systems, including business ecosystems, is that it’s often very difficult to predict the impact of changes. Small changes can easily result in outsized effects due to implicit positive feedback cycles. Also, changes may have no perceivable effect until a tipping point is reached where the scales swing over to the other end. Being the storytelling machines that we are, many of us have explanations on cause and effect that likely are wrong.

Although making a decision, even a bad one, is almost always better than failing to do so, it’s still preferable to decide based on data and evidence rather than based on beliefs and opinions. This is where I see many companies fall short in their processes: in virtually all situations, it’s feasible to run experiments with customers or other stakeholders that provide actionable insights into the consequences of decisions that are being considered. Many feel that it’s impossible to experiment with customers due to the effects it may have on brand, perception, customer relation or the current business. It often is forgotten that the alternative is to instill changes on the ecosystem without any evidence and to simply hope for the best.

So, next time you’re asked to make a decision, start by asking yourself if you have all the necessary information, evidence and data. If the answer is negative, rather than making a blind decision based on opinions, ask yourself what the most informative, simplest-to-execute, lowest-impact and fastest experiment is that you could initiate to collect the data you need to inform your decision.

Some organizations may have a culture where admitting that you don’t know is considered a sign of weakness. However, there’s ample evidence of the consequences of poor decisions, ranging from features in products getting prioritized that never get used by users to entire companies that fail catastrophically. As I wrote in an earlier blog post, you think you know, but you don’t. Whenever you don’t know, admit it and find a way to run experiments to figure out what’s actually true and what’s not.

To get more insights earlier, sign up for my newsletter at jan@janbosch.com or follow me on janbosch.com/blog, LinkedIn (linkedin.com/in/janbosch) or Twitter (@JanBosch).