In recent months, I’ve started a series of online and in-person workshops around Agile. These workshops have focused on the challenges of Agile, the things in Agile we want to keep, the things we want to remove as well as practices that we seek to add. Here, I intend to discuss the key elements of what I believe will be the next paradigm after Agile. However, before we dive into that, I think it behooves us to discuss why this is even a topic in the first place.
The starting point for me was that during 2023, I started to notice that most of the companies I work with claim that they employ agile practices to some extent, but that in practice, these practices seldom went beyond the team level. There are several places where a set of teams operate using agile practices together and have set up a continuous integration and test infrastructure, but at the product, portfolio and company level, nothing has really changed for decades.
Interestingly, many of these companies have adopted SAFe as the overall product development method for software. However, to quote a manager at one of the Software Center partners: “We adopted Agile and nothing changed.” In most SAFe deployments I’ve seen, it very much was an ‘emperor’s new clothes’ situation: lots of smoke and mirrors but no fundamental changes to the business model, product development and other aspects of the business.
Although there always have been Agile detractors, the majority of people I worked with at these companies, as well as I myself, have for the longest time felt that Agile was the right way forward and that the reason why we haven’t come all the way with it is that we haven’t tried hard enough and key decision makers didn’t understand it. Last summer, I even wrote a blog post on the topic.
Over the last year, however, I’ve started to entertain an alternative view: Agile simply is not, or is no longer, the right paradigm for product development, especially in the software-intensive industry. My current hypothesis is that no matter how hard we try to squeeze the square peg into the round hole, it simply isn’t going to fit.
To explore this, we need to go to the basis of Agile. To me, that’s the Agile Manifesto. We’ve all seen it, but it’s important to remember that it was developed by people who, by and large, worked with a single team for an individual customer that the team had continuous access to.
This is far from the reality for most engineers and teams. We’re developing a product for a market consisting of hundreds, if not millions, of customers. We’re in an organization with dozens of teams, all contributing to the product. The product contains mechanical, electronic and software parts. Finally, in many markets, customers are actively resisting continuous monetization and simply want to purchase the product, use it up and buy the next one, meaning that there’s no business model to compensate for the costs associated with DevOps.
The mismatch between the original principles and the reality of product development is where, in my view, things started to go off the rails. We needed to compensate and develop solutions for all the aspects that Agile didn’t solve and where it ran into conflicts. Two easy ones are scale and atoms.
When scaling development organizations, sprint planning with all teams is simply a nightmare. Hence, approaches like Agile focus on quarterly increment planning rather than bi-weekly. That of course is quite close to the waterfall-style ‘iterative development’ approaches that preceded Agile and far from a DevOps style of development.
For products that include atoms, ie mechanics and electronics, we simply have to align with the fact that parts need to be ordered, manufacturing lines need to be planned and set up and that, gasp, we actually need to agree on a date far into the future when everything needs to be ready to avoid inefficiencies. No matter how agile you are, no company is going to set up a manufacturing line in a sprint or continuously evolve the mechanical and electronic parts of the product in response to new features and functionality.
When stepping back and reflecting on the state of practice, my perspective is that many ‘agile’ approaches simply took pre-Agile, waterfall-style techniques to complement Agile where it fell short. This has resulted in the situation where we’ve developed new labels and rituals but in practice, nothing has fundamentally changed.
There’s a group of people, often consultants, who claim that everything we can think of falls under the banner of Agile as, in their view, Agile is continuously evolving into something new. Of course, we can bolt on all kinds of practices, but at the core, there’s a set of Agile principles underlying the entire paradigm that simply are there. Of course, we can call whatever we want “agile,” but it doesn’t mean that we’re still beholden to the paradigm as it was initially intended.
Based on the workshops, several aspects of Agile can be identified that simply don’t work or where we’ve gone overboard. Take the many rituals and time-consuming practices that add little or no value. Another problem is that large efforts that are difficult to plan and implement can’t be completed by one team in one sprint. Aligning agile software with lean and waterfall-oriented mechanics and electronics is difficult as well. When the number of teams becomes large, high coordination costs are incurred. When teams don’t want to commit beyond a single sprint, product releases are impossible to plan. These are just a few of the many, many more concerns workshop participants had, next to the many responses they gave on how they solved some of the challenges while staying within the Agile paradigm.
For the sake of argument, I’m staking out my position: Agile has been great for the software development community. It was defined in the early 2000s as a reaction to the CMMI straightjacket that many older engineers may remember and it served that purpose well. In most contexts, developers have much more freedom than what a CMM level 5 organization would have afforded them. However, we’ve reached the end of the Agile paradigm and we need to move on to something else. This “something else” will maintain many of the elements of Agile, in the same way as Agile maintained elements from the waterfall paradigm before it. However, we need to take a fresh start in how we think about product development, unencumbered by what Agile has to offer. That’s what this series will be about.
The Agile paradigm was defined in 2001 in response to the strict process orientation of CMM. It has served us well for more than two decades, but the limitations of the paradigm are becoming increasingly visible for those who dare to look closely. We need to move on to a new paradigm that will address some of the key limitations of the Agile paradigm while maintaining several of the key principles. This requires a fresh start, rather than an evolution. To quote Christopher Ferry: “A fresh start isn’t a new place. It’s a new mindset.”
Want to read more like this? Sign up for my newsletter at jan@janbosch.com or follow me on janbosch.com/blog, LinkedIn (linkedin.com/in/janbosch) or Twitter (@JanBosch).
