As I am slowly emerging from summer vacation, there are a couple of reflections that have come to mind as I am engaging my brain again. One topic that I have been noodling on is that I have noticed an undercurrent of anti-agile thinking in what I read and the people I talk to. It seems to me that we are reaching an inflection point where the general tendency is moving away from agile.
The question is of course where we are going, but first I think it is important to remind ourselves where we are coming from. Before the advent of agile, two decades ago or so, the software industry focused excessively on process. The idea was that by defining and following a well-defined process, we would solve all the issues in software engineering ranging from lack of predictability to quality issues at customers or late in a development project. The pinnacle of this movement was, in my view, the capability maturity model (CMMi) with its five maturity levels and some companies priding themselves on being assessed to be at the highest level.
The agile manifesto managed to hit the zeitgeist in the 2000s as the general discontent with rigid, straightjacket processes was running high. The principles in the agile manifesto such as “individuals and interactions over processes and tools” and “working code over documentation” resonated with large groups of engineers who felt hampered, rather than empowered, by the processes. The first experiences with working with agile practices were very successful and over time more and more organizations adopted the new ways of working.
Now my perception is that we are moving towards an inflection point where agile is viewed as the problem, rather than the solution. Agile practices were initially intended for small development teams of six to ten people that worked directly with a specific customer to build a bespoke solution for that customer. However, many companies employ dozens, hundreds or even thousands of software engineers. In addition, many companies build products that are sold to a market of business customers or consumers. These realities required the agile practices to be scaled and adjusted to fit the reality of the industry in which the company finds itself. This typically means that roles get inserted between the team and the actual user of the software built by the team such as product managers and product owners, introducing layers of indirection and, consequently, misinterpretation of qualitative feedback.
Although DevOps is increasingly used as a mechanism to get the results of a sprint out to customers, the quantitative feedback loop between the team and the system deployed in the field is often non-existent or highly distorted. When dozens of teams all provide new code as part of the DevOps cycle, it is often exceedingly hard to determine the actual impact of the functionality a team spent the last sprint on.
As neither the qualitative nor the quantitative feedback loop are functioning well in many companies, we return to the traditional model of product managers and others to prioritize and plan work. In my view, the Scaled Agile Framework (SAFe) is an exponent of this notion and even if it has generated lots of revenue for many people, the fact of the matter is that many companies have realized that SAFe did not really change anything. We are no closer to the customer, we are no closer to having continuous feedback on what we are building, we are still using a pretty heavy plan-based process and increment planning, to me, feels very much like a contract negotiation.
A second challenge is that agile principles are oftentimes religiously interpreted and every form of development has to be shoehorned into a standardized process. Major new functionality that requires significant infrastructure development as well as new components to be built may not fit in a single sprint. Similarly, in some contexts, a sprint is simply too long for the work items that the team is working on and they need continuous reprioritization throughout the sprint. Forcing development in different contexts and of various nature all into the same standardized process is causing friction and frustration in the same way that CMMi did.
Although I will not make any predictions on what is next after agile, if the inflection point really is here, but I do want to share three principles that would be good to stay aware of, i.e. BAPO, feedback loops and uncertainty.
First, the BAPO model states that business and business strategy should drive architecture and technology choices. These should in turn drive process, ways of working and tools. Finally, these should drive organization. Although the BAPO model was published two decades ago, I still see many organizations fall into the trap of starting from process or organization and looking to address business and architecture from that point. Process is not the anchoring point, but rather should respond and be adopted to the business and architecture that the company is pursuing.
Second, organizations, teams and individuals need feedback loops. We need to know what the consequences of our efforts are. One of my managers at Nokia always said that effort is not the same as progress. In many organizations, people are exceedingly busy, but they are not necessarily creating a lot of progress for the business. This easily leads to internal shadow beliefs where the entire organization believes that some goal needs to be accomplished in order to serve the customer, but when it is accomplished there is no positive impact whatsoever. As I have mentioned before, our research shows that half or more of all the features built are never used or used so seldomly that these should not have been built in the first place
Third, humans hate uncertainty and organizations even more so. We always look for ways to reduce, manage and control uncertainty, despite the futility of it. We create plans, have meetings to align plans from different teams and organizational units, make projections on the impact of our efforts, etc. all with the intent of reducing uncertainty. The reality is, of course, that many things simply are unknowable. There is no way to know until you try it out. So, rather than trying to plan and project, simply adopt the notion of experimentation. We need to try out things with fast, cheap experiments, learn and adjust the next step.
Concluding, in my view, agile has brought a lot of good to the software industry and with the criticism that it is receiving today, we have to be careful to not throw out the baby with the bathwater. Fast cycles are preferable in the vast majority of cases. When you manage quality, customers want their systems to become better continuously through software updates. By admitting that you don’t know, you can adopt experimental practices, such as A/B testing, that allow you to at least double the effectiveness of your R&D in terms of value created. Whatever comes after agile, these principles, in my view, need to be integral parts of the next paradigm for software development. As Thomas Kuhn, the author of the famous book on scientific revolutions, said: “we view the world in terms of our theories.” So, let’s make sure we adopt the right ones.
Want to read more like this? Sign up for my newsletter at jan@janbosch.com or follow me on janbosch.com/blog, LinkedIn (linkedin.com/in/janbosch), Medium or Twitter (@JanBosch).
