Humans hate uncertainty. Even saying “I don’t know” feels very uncomfortable for many. There are very strong evolutionary drivers for this as our prehistoric selves experienced existential risk whenever they entered situations with high uncertainty. As a consequence, we organize our lives to maximize certainty, ranging from the houses we live in, the jobs we select and the vacations we take. Everyone who says they like surprises is lying through their teeth. People only like positive surprises. We only like it when things turn out better than expected.
One of the irrefutable aspects of life is that the future is fundamentally unpredictable. We simply don’t know what’s going to happen tomorrow. Even if the COVID-19 pandemic is starting to become a bit of a memory, I think that many remember being completely taken by surprise as to how one stupid virus could disrupt society and our lives to such an extent.
Whenever something unexpected happens, our first reaction is to try to explain how it could have happened. My favorite example is news reporting on the stock market. Even though the stocks on average go up all the time and there are very few 10-year periods where they didn’t end up in positive territory, the day-to-day movements in the market are largely random. However, that’s such an unsatisfactory idea that we demand other answers. Hence, we watch people on television explaining to us what happened and why, even though they’re talking complete rubbish. It simply is post-rationalization that removes cognitive dissonance.
Until the Enlightenment in the 17th and 18th centuries, the primary answer was to use religion. In virtually all religions, unexpected things, both negative and positive, are attributed to the will of some god. As we’re eager to take action to reduce uncertainty, making offerings to the gods is a way to gain their favor. I find it hilarious to see people laugh at the antics of our ancestors while engaging with full commitment in corporate planning activities that often are based on such a shaky set of assumptions that we might as well take a chimpanzee to throw darts at the wall. In many ways, we’ve replaced the old religions with new ones without calling these as such, simply because we can’t stand dealing with uncertainty.
In product management, uncertainty is even more prevalent. All decisions we make are based on an assumption or prediction of the outcome or consequence of a decision. Whenever we decide to add a feature, build a new product or enter a new market, we’re doing so based on a prediction of the effect of doing so. However, our predictions have a very high likelihood of being off to some extent.
Especially in companies where many are involved in the decision-making process around R&D investments, we can’t simply place our bets by ourselves. We also need to convince others that the initiatives we propose will have the expected impact and are the highest-impact things we can do. The most effective strategy is to project certainty, which results in us creating an illusion of certainty that in many companies tends to become a self-fulfilling prophecy. When successful, everyone has drunk the Kool-aid and we all are convinced that the expected outcomes are 100 percent going to happen. Until reality catches up and kicks us in the shins.
Obviously, this approach, though widely prevalent, is very inefficient as significant amounts of R&D resources are wasted on initiatives that have no or only negative impact. Instead, we need to embrace uncertainty and use constructive techniques to systematically and iteratively reduce it. The primary mechanism is to simply use the scientific method, ie hypotheses and experiments, combined with fast feedback cycles.
Those who have followed my writing for a while know that I’m not a fan of requirements. The primary concern is that the one specifying the requirement implicitly claims that building the functionality realizing the requirement will predictably lead to positive outcomes. As research by us and others shows that somewhere between half and two-thirds of all features in a typical system are never or hardly ever used, this is obviously not the case. Instead, we should adopt the notion of hypotheses. A hypothesis is a testable statement of the “if this then that” type. In R&D, this often takes the form of “if we build this, that will be the effect.”
The advantage of using hypotheses is that we can organize R&D as a set of bets on outcomes. This allows us to balance big bets that are high risk and high return with small bets that are low risk but obviously have less return. In addition, simply using the terminology of “hypothesis” instead of “requirement” captures the uncertainty and keeps it front of mind for everyone involved.
Viewing R&D initiatives as experiments to achieve a certain outcome has the same benefits as using the term “hypothesis.” An experiment is intended to test a hypothesis in a reliable, preferably statistically validated, fashion. Hence, organizing units of R&D work as experiments ensures that we keep a very strong connection to the intended outcome. It facilitates the conversation between product managers and engineers and allows more minds to engage around the problem area.
The third mechanism is fast feedback cycles. The general principle should be that the more uncertain the outcome of a specific course of action is, the shorter we should aim for the feedback cycle to be. For instance, many software companies traditionally used annual release cycles, which meant that the “bets” the company was placing were measured in dozens or hundreds of person-years. With the adoption of DevOps, we’re shortening the feedback cycle to some person-weeks of effort between customer proof points. Fast feedback allows us to move away from bad ideas much earlier than yearly release cycles. This is where A/B testing shines as it allows us to rapidly have quantitative feedback on our hypotheses.
There are of course situations where a large and long-term investment is unavoidable or at least difficult to avoid. In the automotive industry, for example, new generations of platforms tend to be developed every five to seven years and the required investment easily runs into the hundreds of millions if not more than a billion. However, even in these situations, we can triage the design decisions we need to take into those that by necessity need to be taken early and under high degrees of uncertainty and those that can be delayed and hopefully tested before being made permanent. That brings us to the notion of reversible and irreversible decisions that we’ll return to later in this series.
Humans abhor uncertainty and many of our behaviors are concerned with reducing it. This can easily lead to an illusion of certainty, which is dangerous as we start to ignore the risks associated with the actions we commit to take. Instead, especially in product management, we need to embrace uncertainty, use the notions of hypotheses and experiments instead of requirements and seek to shorten feedback cycles as much as possible. To quote Voltaire, “Doubt is uncomfortable, certainty is ridiculous.”
Want to read more like this? Sign up for my newsletter at jan@janbosch.com or follow me on janbosch.com/blog, LinkedIn (linkedin.com/in/janbosch), Medium or Twitter (@JanBosch).
