Introduction to Data
The ability to continuously deploy new versions of software to some or all customers leads to another major evolution concerning the radically changed ability of companies to measure how users are using the software. Although software companies have often used technique to collect usage and quality data, continuous, deployment demands continuous connectivity for the deployed systems. This, in turn, allows for the collection of much more detailed, timely data.
The ability to collect product data in real-time leads to the possibility to significantly shorten the feedback loop of product development. Shorter product development feedback loops allow for a much more rapid discovery of the functionality and qualities that lead to significantly increased alignment between the needs of customers and the product. This in turn improves the competitive position of a company as the product better meets the market demands.
The short feedback loops in product development and evolution allow for a different approach to requirement engineering: the deployed system can now be used for experimenting with customers and with system behavior. Different deployed systems can get different versions of the software or of feature implementations. By studying the behavior of users and systems while using the different versions. one can empirically and based on statistically relevant data determine which version of a feature implementation or system software is preferred. As many studies have shown the major gap between espoused preferences and actual behavior, the ability to collect data on actual behaviors incredibly valuable.
Data Dimension in Short
The way companies evolve through their ability to use data proves to follow a predictable pattern. As shown in figure below, companies start with a very ad-hoc (level 1) and manual approach (level 2) to data driven by passionate and committed individuals.
Then, when the collection of data is automated (level 3), followed by the introduction of dashboards for relevant teams that get automatically updated with data from the field. The challenge with these dashboards is that they easily become outdated, which leads to the data innovation stage (level 4) where there is a constant flow of new insights that results in evolving dashboard and focus areas.
Finally , the entire company adopts data-driven decision making (level 5) in everything it does including sales, performance reviews, hiring and other processes.