On Hierarchy, Micro-Management and Leadership in R&D

Over the last years, I periodically visited a company that I have really admired for a long time for their focus on autonomous agile teams. These teams have a lot of freedom within their area of responsibility and can choose virtually anything from the platforms to use, the tools to use, when to release, what functionality to build, etc. The company is data driven and the teams get a lot of energy out of knowing that they were contributing to the revenue of the company as there were metrics in place that allowed teams to estimate their contribution.

Recently I visited them again and learned that several of the people that I had talked to over the last year either had left, were in the process of leaving or were planning to leave. So, what happened? Why this sudden change of heart among the R&D staff? It turned out that new senior R&D management had come in. They had analysed the situation, determined that freedom that the R&D teams experienced was causing major inefficiencies as there was no sharing of commonly needed assets, no focus on cost and lots of duplication. In response to this conclusion, they centralized decision making and used the reporting hierarchy to enforce a new set of decisions and consequent behaviours. The result was that attrition went through the roof as many felt that the implicit contract between the company and its employees was broken. And in a pretty hot market for engineers, finding another exciting opportunity is not difficult at all.

So, what can we learn from this? The wrong conclusion is that R&D management should have left the teams alone. Especially in fast growing companies, there are points where a course correction in the way that R&D is conducted is required. Continuing with the “old” way of doing things even though the company needs a “new” way of doing things is as risky, if not more so, than not changing. So, the question is how to realize this course correction without all the negative consequences described above.

The R&D management at the aforementioned company fell back on the only mechanism they new: traditional, hierarchical management, centralization of power and micro-management to enforce the new behaviour. Of course, this means dis-empowering teams and individuals and treating them like replaceable cogs in a machine. This is extremely demotivating for engineers and this causes those who have a clear idea of how they want to work and that have alternatives available to seek greener pastures.

One of the main characteristics of agile development is that it empowers teams and individuals to perform their work in the way that the team considers best. As such, agile development is at the forefront of a major transition in industry towards self-managed teams and organizations. In our research (see reference below), we have studied how organizations adopt empowerment and self-management. As shown in the figure below, organizations transition from traditional to empowered organizations through a number of steps.

selfmanagedhierarchy

The table shows how self-management and empowerment starts in teams and then spreads throughout the organization until even the culture in the company is infused with this kind of thinking.

selfmanagedtable

In this new world of empowerment and self-management, the use of hierarchy and micro-management is not longer the strategy of choice. If you fall back to that, your people will just leave you. Instead we need leadership. We need leaders who can explain to and convince teams why changes are necessary and what the road is that we need to follow to get there. And once the goal and path are clear, the teams need to be empowered to figure out how to get there and the leaders become coaches and mentors. We need to achieve self-managed and empowered organizational change.

If this piece resonated, please share your thoughts with me as I am always looking for more examples and cases to learn from!

Reference:
Olsson, H.H., and Bosch, J. (2016). No More Bosses? A multi-case study on the emerging use of non-hierarchical principles in large-scale software development. In Proceedings of the 17th International Conference on Product-Focused Software Process Improvement (PROFES), November 22nd-24th, Trondheim, Norway.