{"id":1164,"date":"2020-11-17T09:33:04","date_gmt":"2020-11-17T09:33:04","guid":{"rendered":"http:\/\/janbosch.com\/blog\/?p=1164"},"modified":"2020-11-17T09:33:10","modified_gmt":"2020-11-17T09:33:10","slug":"so-much-data-so-little-value","status":"publish","type":"post","link":"https:\/\/janbosch.com\/blog\/index.php\/2020\/11\/17\/so-much-data-so-little-value\/","title":{"rendered":"So much data, so little value"},"content":{"rendered":"\n
\"\"
Image by Gerd Altmann from Pixabay<\/figcaption><\/figure>\n\n\n\n

Recently, in a discussion with a company about becoming data-driven, I ran into the same challenge as many times before: the company claims to gather so much data, but the amount of value generated from that data is very small. It makes one wonder what underlies these patterns of, apparently, enormous amounts of data being collected but very little of that data being used to create something of value. In my experience, there are at least three factors at play: sense of ownership, local optimization and cost of \u2018productizing\u2019 data.<\/p>\n\n\n\n

A typical pattern in many organizations is that teams generating and \ncollecting data for their purposes feel strong ownership of that data \nand don\u2019t want others to prod \u201ctheir data\u201d with their big, fat fingers. \nIt\u2019s theirs and if anyone else needs similar data, they can go and \ncollect it themselves rather than get it for free from the team.<\/p>\n\n\n\n

This leads to many small islands of data that are entirely \ndisconnected and don\u2019t aggregate into something more valuable than the \nsum of the parts. Teams may brag about all their data, but nobody else \ncan use it.<\/p>\n\n\n\n

Any team that decides that they need data to improve the quality of \ntheir decisions will focus on their own challenge and only collect what \nthey need at the level of detail, frequency and aggregation that they \nneed. In addition, they can decide on a moment\u2019s notice to fundamentally\n change the way data is collected, as well as what data is collected.<\/p>\n\n\n\n

The consequence is that the data typically is hard to use outside of \nthe immediate context for which it was generated. This leads to \ndifferent teams collecting very similar data, due to the lack of \ncoordination. Also, as few think about the broader use, teams that \nrealize that they need data are unable to reuse any of the existing data\n as it\u2019s so specific to the use case for which it was collected.<\/p>\n\n\n\n

If a team would decide to make their data available for others, they \nwould need to provide documentation on the semantics of the data, set up\n a system for finding and downloading data sets, ensure that changes to \nthe way data is collected, the semantics, and so on, are carefully \ncommunicated to stakeholders and, of course, respond to requests from \nthese stakeholders and make changes to the data collection processes not\n to benefit themselves, but to help others in the organization. And, \nlast but not least, the team may easily be held accountable for privacy,\n GDPR, security and other concerns that companies have around the stored\n data.<\/p>\n\n\n\n

The consequence is that, unless a counterforce is present, teams will\n actively try to not share data because of the effort and cost of \nsharing with others in the organization. This again leads to lots of \ndata recorded, stored and used for specific, narrow use cases, but no \nsynergies, no end-to-end understanding of systems in the field and the \nway customers are using it, and so on.<\/p>\n\n\n\n

The solution to these challenges is to adopt a hierarchical value \nmodeling approach where you connect top-down business KPIs to \nlower-level metrics that can be collected directly from the field. By \nbuilding this hierarchical, acyclic, directed graph and quantitatively \nestablishing the relationship between higher and lower-level factors, we\n can finally start to generate business value from all the data we \ncollect.<\/p>\n\n\n\n

Getting from the current state to this hierarchical value model isn\u2019t easy, if only because most people in the companies I work with find it extremely hard to determine what quantitative factors we\u2019re optimizing for, and if we do know, the relative priority of these factors is a source of significant debate. However, it provides enormous benefits as you can focus data collection on the things that matter, use the data to make higher-quality decisions and build data-driven offerings to customers that you couldn\u2019t have created otherwise. As the adage goes, it\u2019s not about what you have, but about how you use it!<\/p>\n\n\n\n

To get more insights earlier, sign up for my newsletter at <\/em>jan@janbosch.com<\/em><\/a> or follow me on<\/em> janbosch.com\/blog<\/em><\/a>, LinkedIn (<\/em>linkedin.com\/in\/janbosch<\/em><\/a>) or Twitter (<\/em>@JanBosch<\/em><\/a>).<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

Recently, in a discussion with a company about becoming data-driven, I ran into the same challenge as many times before: the company claims to gather so much data, but the amount of value generated from that data is very small. It makes one wonder what underlies these patterns of, apparently, enormous amounts of data being … Read more<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"generate_page_header":"","footnotes":""},"categories":[4,10],"tags":[],"_links":{"self":[{"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1164"}],"collection":[{"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=1164"}],"version-history":[{"count":1,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1164\/revisions"}],"predecessor-version":[{"id":1166,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1164\/revisions\/1166"}],"wp:attachment":[{"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=1164"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=1164"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=1164"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}