The first blog post in this series discussed the challenges that people are experiencing when working with the agile paradigm. In general, there are three main areas that Agile considers out of scope and that, in my view, need to be included in the succeeding paradigm. These areas are product management, systems engineering and scaling, including architecture, data, organization and business.
Based on the research I’ve conducted with colleagues over the last decade as well as the input of online and in-person workshops, I’ve tried to put together a strawman proposal of what a next paradigm might include and entail. The working name of the proposal is an acronym as well as a word: Radical. I’ll start by giving a high-level overview of the concepts and dive into details in future posts.
R: Responsive
In essence, Agile is a reactive paradigm and much of the conversation is about responding to change. The basic notion is that we exist in some kind of steady state, there’s an external disruption that requires us to react and we use agility to return to the steady state. Being responsive is different in that there’s a strong sense of direction and intended destination. When there are external forces, we respond by incorporating these forces, without losing the direction we’ve set.
Three aspects of responsiveness deserve to be discussed: vision and definition of success, triaging external forces and redefining success. In short, we need to know where we’re going and how we plan to get there. Then, when disruptions or triggers occur, we need to know which ones need to be responded to and which ones can be ignored. Finally, there are times when the initial destination is no longer the best one and we need to redefine it.
A: Architecture
As Agile was in part a response to CMM and process-heavy approaches with significant focus on requirements and architecture rather than code, the two have always been at odds. Also, the Agile Manifesto prioritizes human interaction. The challenge is that coordination through meetings, which is what human interaction often is about, is orders of magnitude less efficient than coordination through architectural interfaces. As long as teams stay on their side of the API, there’s no need to interact.
Architecture in many ways is a highly overloaded term, but here, we’ll focus on three aspects: the relation between business strategy and architecture, system and software architecture and architecture refactoring. Many engineers fail to recognize the, sometimes enormous, implications of architecture design decisions on business strategy options. Second, especially for systems including mechanical and electronic components or where an externally enforced architecture constrains architecture choices, we need to clearly define interfaces at multiple levels, including technical, ways of working and organization. Finally, like the rest of the system, architectures require continuous evolution and this evolution needs to be managed through architecture refactoring.
D: Data-driven
For all the terabytes and petabytes that they’ve stored, most decisions in the companies I work with are still opinion-based. Where conflicts with the data exist, the data is reinterpreted to match opinions. We need to invert this way of operating and ensure that we start from the data and proceed from there to decisions where we allow qualitative data and ‘bets’ to enter the process.
We can take several perspectives concerning data, but here, we focus on three: defining outcomes quantitatively, using experimentation to manage uncertainty and data management. We need to define the intended outcomes of actions in the company in quantitative terms to make sure that we’re making real progress. Second, uncertainty is a fact of life in virtually anything we do. We can use experimentation to decrease uncertainty with limited use of resources. Finally, the infrastructure we use for collecting, processing and storing data as well as managing data assets requires careful design by the company.
I: Integrated cross-functional alignment
Agile never addressed the organizational issues we experienced between ‘the business,’ including product management, and R&D. Experience shows that to be agile, we need to focus on cross-functional teams that incorporate all the necessary functions to respond and execute quickly. One of the main reasons is that intra-team coordination is an order, if not several orders, of magnitude more effective and efficient than inter-team or even inter-organizational coordination.
Few topics receive as much heated debate as organization and I could fill volumes about it. To maintain conciseness and focus, we’ll limit ourselves to three main aspects: the nature of cross-functional teams, steering cross-functional teams and the link between architecture and teams.
C: Continuous
Many of the companies I work with still organize R&D in projects. This means that each release of a system version is often treated as a project and there’s no notion of continuity around a specific product. Earlier, I discussed that companies evolve from being project-centric to being product-centric and then platform-centric. This is even more important in a situation where products are expected to be continuously updated (DevOps) based on data from the field (DataOps), including the improvement of machine learning models (AI/MLOps). If the evolution of the product is supposed to be continuous, development has to be as well.
Although everyone I talk to in the companies I work with agrees in principle, many raise concerns about their specific situation, claiming that they’re different and that these principles don’t apply to them. In this series, we’ll focus on three main arguments for why companies don’t apply continuous practices. First, one frequent argument is that the customer doesn’t want continuous deployment. Second, we’ll look at the business model discouraging continuous practices. Third, we’ll discuss systems engineering in the context of continuous ways of working as in many contexts, the argument is used that we can’t work continuously because of the limitations of mechanics and electronics.
A: Aligned on business goals
One of the realizations that never ceases to amaze me is that so few organizations are aligned on their business goals. Often, there’s some high-level veneer of alignment, but when we start to drill down, it rapidly becomes clear that different departments, functions, teams and sometimes even individuals optimize for different factors. These can even conflict with each other, either because of local optimization or due to malicious behaviors that, in some way, are accepted within the corporate culture.
We’ll again focus on three areas concerning business goals. First, we need to discuss what companies optimize for anyway as many have fallen into the worthwhile-many-versus-vital-few trap. Second, we need to explore how teams can be connected to quantitatively defined business goals and how we can determine the success, or lack thereof, of teams. Finally, we’ll look into the evolution of business goals as companies, in different stages of maturity and in response to changes in their business ecosystem, need to evolve their strategy.
L: Leadership
Oh, the famous L word. So overloaded and so filled with hyperbole. And yet, companies need leadership to align teams, groups as well as the entire organization around a single goal and vision. It’s incredibly difficult to navigate the organizational landscape and exercise leadership effectively as a formal leader. As many organizations are increasingly relying on informal leaders through architects, community leaders and champions, the challenge is even harder for those roles.
As the final topic, we’ll wade into the morass that is leadership and explore some of the key topics. The first is organizational culture. Leaders are responsible for creating the right culture and then ensuring it is real and not a paper tiger that people pay lip service to. The second topic we need to touch upon is the notion of purpose. Especially the new generations expect and even demand that companies have a purpose beyond earning money. It’s the role of leaders to define and clarify the purpose of the company and then ensure that everyone acts in line with it. Finally, we’ll discuss conflict. For all kinds of reasons, conflicts will surface in companies and failing to deal with them will often create confusion, complexities and passive-aggressive behavior that, over time, become increasingly large backpacks that the organization is dragging around. As you might have guessed, it’s up to leaders to resolve conflicts and get the dissenters to the ‘disagree and commit’ point.
As we reflect on the Agile paradigm, we can see that it served a worthy purpose when it was first introduced, but more recently, it’s excluding several critical dimensions that need to be incorporated. As a strawman proposal to start a conversation around this, I propose Radical as the next paradigm. It incorporates many aspects of Agile but lets go of some and adds several new perspectives. Although some may view this as a useless exercise, I want to end with a quote by Stephen Covey: “Paradigms are powerful as they create the lens through which we view the world.”
Want to read more like this? Sign up for my newsletter at jan@janbosch.com or follow me on janbosch.com/blog, LinkedIn (linkedin.com/in/janbosch) or Twitter (@JanBosch).
