The End Of System Architects

The majority of companies that I work with provide products, systems and solutions to the market that include mechanical, electronic (hardware) and software parts. In order to design these systems, companies appoint system architects that are supposed to manage the system design, realization and evolution. Over the last couple of months, I have run into the same problem at multiple companies: system architects don’t understand software!

Most system architects have a background in mechanics or electronics and try to treat software as if it’s made of atoms instead of bits. In fact, I hypothesize, based on significant circumstantial evidence, that many system architects have a deep seated dislike of software, of developers and of software architects and try to spend as little time with this pesky subject. The consequence of this attitude is that system architects tend to make really poor choices concerning mechanical and hardware architecture that cause negative and sometimes disastrous effects on software.

Some of the problems that I have observed in the various companies that I work with have to do with meaningless mechanical and hardware variants without business value, system architectures with high coupling, a complete disregard of backward compatibility during system evolution, lack of abstraction mechanisms and a general tendency to blame software for everything that is wrong with the system. Below I describe some of these problems in more detail.

Most companies support dozens if not more variants of mechanical parts and hardware architectures that have no or minimal business or customer value. Many system architects seem to prefer to create unique solutions for every physical product that they get to design. This leads to a large number of variants that all need to be supported by the software. There is significant cost associated with new physical parts and hardware architectures in and of themselves, but the resulting software complexity due to variant handling and the complex dependencies can easily be on par in terms of cost and time to market implications. We need to be much more business driven in our decisions and seek to limit the set of mechanical and hardware variants to the absolutely minimal required to deliver optimal customer value.

Deeply integrated and highly coupled architectures cause numerous ripple effects when changes occur. Many system architects are hell-bent on minimizing the bill of materials for the physical part of the product and design lots of cross-system dependencies into the architecture as it allows for maximizing the performance of the hardware components. The consequence is that any change to any part of the hardware immediately has ripple effects throughout the system architecture. The software, which depends on the interfaces provided by the hardware, breaks frequently and the maintenance cost is much higher than in the case of a system design that would incorporate the needs of software into its constraints.

The problems are exacerbated by the lack of understanding of the importance of backward compatibility. Rather than introducing a new interface while maintaining the previous version for a period of time, the introduction of a new version of hardware brings several breaking changes with it. And as the software will be expected to support all the versions of the hardware as well as be reusable across the product portfolio, the set of variants to support, and the associated cost and effort, only grows.

The highly coupled, deeply integrated architectures also cause a lack of abstraction layers, forcing the software to compensate for the exact sensor types, actuators types, etc. in the specific product version. In some companies that I work with, the exact sensor, actuator and other hardware part needs to be exposed all the way up to the user interface! As if the user cares about the specific hardware part version and its capabilities!

Finally, several system architects have a tendency to blame software for everything that goes wrong in a project. After a development cycle where first the mechanical part of the system is designed and runs late, it is followed by a design of the hardware which then runs late as well. By then the software organization knows what to design for, but of course has very little time left due to the time overruns by the other groups. Still trying to do the best they can, the teams skimp on features and quality assurance in order to help the company meet its deadlines. The result is that software is always late and has poor quality. However, rather than blaming the software organization, it would be good to realize that the root causes of these problems originate upstream.

The only solution that I have been able to come up with is that we have to get rid of system architects. At least, we need to get rid of the system architects that have a mechanical or electronics background, rather than a software background. Instead, we need to promote software architects to the role of system architect. Software architects learn to deal with complexity through the use of abstraction and decoupling from day one in their career. In addition, software architects, in my experience, have a better understanding of the business than other types of architects. Finally, in most companies, the mechanics and electronics (the “atoms”) have commoditized and the differentiation has shifted to software (the “bits”). Hence, to maximize the value that we deliver to customers, we need to put the architect in charge whose expertise is closest to the functionality customers actually care about. Taking this perspective will allow us to reduce variants, freeze investment in mechanics and electronics that are commodity, find ways to outsource relevant parts of the systems engineering and to focus our resources on the differentiation provided by software.

Finally, the point I make in this article is rather strongly worded. The reason is that I see many companies and their employees agree with my line of reasoning, but fail to do something about it. Product managers feel that they are victims of the system architects, the software folks have reached a state of “learned self helplessness” and business folks try to monetize the systems and products they get from R&D. The time has come to make a material shift in the way we think about systems engineering, who we appoint as systems architects and how we align systems architecture with the business strategy. We live in a software driven world and it’s about time we start acting like it!