How to develop software

In earlier posts (such as here), I have discussed different approaches to software development and one of the key models for this is the HoliDev model where we combine requirements driven development, outcome-driven development (e.g. A/B testing) and AI-driven development. In the figure below, the HoliDev model is shown graphically.

It may easily seem that for each feature or functionality, one type of development is selected and then used throughout its lifecycle. In practice, however, we see that the development approach evolves over its lifetime. For instance, one could imagine a development process where functionality evolves in the following way. First, product management has determined that a certain function or feature is relevant for the customer. This function or feature is initially specified in a requirement specification and then built by R&D. Even though it may not be clear what the metrics are that the function or feature is looking to improve, but it is reasonable to assume that there are implicit assumptions about the effect of the function or feature on the performance of the system.

After deployment, the instrumentation of the system will show certain changes in the metrics and KPIs that are collected that can, reasonably, be attributed to the newly added function or feature. Assuming the function or feature proves to be valuable, the next step is to consider how we can improve the metrics and KPIs driven by the function or feature. At this point, the team can decide to switch from requirement-driven to outcome-driven development. At this point, the team starts to develop hypotheses and to test these through experimental development approaches, such A/B testing or Multi-Armed Bandits (MAB) algorithms. The team, over the course of several experiments, can optimize the metrics and KPIs by testing the alternatives in operation.

Once the return on investment (RoI) of A/B testing starts to decline, the team can decide to move to the next stage of using AI techniques, such as automated experimentation using machine- and deep-learning (ML/DL) approaches, to drive further improvement. The good news is that at this point is that (1) lots of data has been collected and (2) A/B testing has generated alternatives with different outcomes that can be used as a basis for training the ML/DL system. At this point, the team can put a ML/DL model in place that uses its training and new data as a basis for fully automated improvement of the function or feature over time.

Here we described the process of moving from requirements-driven to outcome-driven to AI-driven development. However, in practice, one does not have to follow this process. Instead, one can start with outcome-driven or AI-driven development directly and evolve it over time.

Similarly, we can move back from AI-driven and outcome-driven development to requirements-driven development when functionality is commoditizing. When it is commoditized, there is no value in driving further improvement and functionality. So, at this stage, the functionality is frozen and no longer being improved upon. However, sometimes there is a need to make some changes to the commoditized component anyway. For example, a defect is found or a dependency on external software ceases to work correctly. In this case, a requirement to update the commoditized component with the intent of keeping the functionality up to date will enter the system.

The key point is that the type of development approach that I select for developing or evolving functionality depends on several factors and there is no fixed recipe for this. Instead, teams should carefully evaluate the benefits and disadvantages for each approach, based on the level of differentiation provided by the functionality, the understanding of the intended effect of the functionality in quantitative terms, the amount of data available, the willingness of customers to be exposed to evolving, periodically changing functionality, etc.
Concluding, there are several types of development of which we outline three in this article. Teams need to select the best type for each feature or system functionality and then evolve to adopt the next type at each iteration. Doing so will allow you to significantly increase the effectiveness of your R&D investments as you have quantitative data to steer decision making.

To get more insights earlier, sign up for my newsletter atjan@janbosch.com or follow me on janbosch.com/blog, LinkedIn (linkedin.com/in/janbosch) or Twitter (@JanBosch).