In my experience, companies tend to fall into two categories when it comes to technology. Either they’re extremely market and customer-focused and view R&D as a necessary evil required to get the products we need to have anything to sell. Or they’re technology-focused and have the implicit belief that as long as we put bleeding-edge technology in our products, customers will buy what we have to offer.
In the first category, R&D often is the scapegoat for anything that goes wrong in the company, ranging from lacking product features to quality issues and from slow responsiveness to being the bottleneck that stops anything interesting and fun. In the second category, it often is the sales organization that’s considered to be a bunch of idiots because they fail to sell the amazing products we build.
In my view, this bifurcation into what to build and how to build it – product management vs R&D – has resulted in extremely poor R&D effectiveness, as expressed in the amount of revenue generated per invested unit of R&D. Research by us and others shows that somewhere between half and two-thirds of all new features in products are never used or used so seldomly that the R&D investment wasn’t worth it. In addition, according to our data, companies spend 70-90 percent of their R&D resources on commodity functionality.
I, therefore, believe that the traditional distinction between product management focusing on what to build and R&D focusing on how to build it is outdated. We need to move to ways of working where product management and R&D professionals work in the same cross-functional team and prioritize work together.
The key reason why this is important is that we have fast feedback loops available between customers and our R&D activities. Thanks to DevOps, DataOps and MLOps (insert your favorite type of Ops), we can within days and weeks receive feedback on the things we worked on during the last sprint. That allows for experimentation and trying things out in ways that were completely impossible earlier.
We can identify three levels of experimentation: iterative development, A/B testing and reinforcement learning. Iterative development is concerned with building the thinnest possible slice of a new feature that we consider to have the largest impact on the value delivered to customers. We explicitly define the expected impact we aim to have, deploy the slice, measure the impact and then decide whether to continue development or remove the feature of the system when it becomes clear that the expected impact isn’t realized. This approach is codified in the Hypex model.
A/B testing is also known as split testing. In cases where it’s not clear what the right way of realizing a feature is, we build it in two or more different ways, deploy all alternatives and randomly assign these to different users to measure which of them leads to the best outcomes. There’s much more to A/B testing than what I cover here.
In reinforcement learning (RL), we deploy an RL model together with a state space, an action space and a reward function. Of course, the model only addresses a small part of the system’s functionality, but in the selected scope, it can experiment with different actions to learn what the best action to take is in each situation (or state). The advantages of RL are threefold. First, the system autonomously learns the optimal behavior without human involvement. Second, it allows for mass customization as each instance of the system may optimize differently based on interaction with the customer. Finally, when things change over time, the RL model will automatically adjust itself to the new reality and find a new optimum.
The essence of experimentation is that a team has to agree on what to experiment on and how to conduct the experiment in the best way possible. This is where product management and R&D come together. Joint exploration allows for the optimization of the experiments to run in terms of expected value as well as the associated R&D cost for realizing the experiment.
In addition to using experimentation to ensure we build new functionality that’s actually used by and delivers value to users, we also need to invest resources into quality assurance and defect management as well as architecture refactoring to ensure the product or platform stays up-to-date. Again, the prioritization of defects and refactorings as well as the timing of these activities over a series of sprints is much more easily achieved when product management and R&D work closely together.
In large organizations where we have many teams, we need to find ways to scale this approach. Again, there are multiple alternatives besides the top-level split between product management and R&D. One structuring principle can be the architecture where cross-functional teams containing product management and R&D skills are responsible for a subsystem or set of components. An alternative approach is to assign responsibility for specific KPIs to teams. In this case, there may be a revenue team, a customer satisfaction team, a new customer acquisition team, and so on. Each of them can make changes anywhere in the product as long as they don’t exceed the guardrail metrics that are owned by the other teams.
Traditionally, product management focused on market needs and business strategy and defined what to build, often expressed in a requirements specification. R&D was focused on how to build it, took the requirement specification and developed, tested and delivered the product. Although that approach works, it leads to very low R&D effectiveness. With new approaches such as DevOps and fast feedback cycles available, product management and R&D need to be integrated into one cross-functional team that focuses on experimentation as well as balancing new functionality, defect management and architecture refactoring to ensure the long-term value of products and platforms. To end with a quote from Ron Kaufman: “First be effective, then be efficient.”
Want to read more like this? Sign up for my newsletter at jan@janbosch.com or follow me on janbosch.com/blog, LinkedIn (linkedin.com/in/janbosch) or X (@JanBosch).
