{"id":946,"date":"2019-08-21T19:52:30","date_gmt":"2019-08-21T19:52:30","guid":{"rendered":"http:\/\/janbosch.com\/blog\/?p=946"},"modified":"2019-08-21T19:52:32","modified_gmt":"2019-08-21T19:52:32","slug":"quantify-yourself","status":"publish","type":"post","link":"https:\/\/janbosch.com\/blog\/index.php\/2019\/08\/21\/quantify-yourself\/","title":{"rendered":"Quantify yourself"},"content":{"rendered":"\n
\"\"
Photo by Markus Spiske on Unsplash<\/figcaption><\/figure>\n\n\n\n

Having spent quite a bit of this summer thinking about machine learning and artificial intelligence, it seems to me that there\u2019s a very important transformation ongoing from a focus on the qualitative to a focus on the quantitative. The moment we start with A\/B testing, deploying multi-armed bandits or training machine learning models, the very first action we need to take is to define, in precise, quantitative terms, what the factors are that we are optimizing for and what the relative priority of these factors is. And, of course, what factors aren\u2019t allowed to change outside a certain set of boundaries.<\/p>\n\n\n\n

In \nmany ways, this isn\u2019t the first time we\u2019re moving into this direction. \nEarlier, the notion of key performance indicators was widely used to \ncontrol teams, departments and companies. Google and, before that, Intel\n have made extensive use of the OKR mechanism (Objective & Key \nResults), which combines a qualitative objective with quantitative key \nresults.<\/p>\n\n\n\n

Still, it seems to me that there\u2019s at least one real \nchange between earlier initiatives and today\u2019s trend and this is the way\n we interact with software-intensive systems. Until now \u2013 and this still\n is the primary way of working \u2013 we\u2019ve developed systems by defining how\n these should conduct their operations. Using requirement specifications\n and similar techniques, we\u2019d describe what the system was expected to \ndo in qualitative terms and then design it and define how it should \naccomplish the intended goal.<\/p>\n\n\n\n

The primary difference now is that \nwe tell systems what to accomplish in quantitative terms, give it a \nbunch of data to train on and then expect it to figure out how to \naccomplish the desired outcome. It\u2019s a bit black-and-white as we still \nneed to provide machine learning or deep learning models, but the nuts \nand bolts of how the system accomplishes the outcomes is through \ntraining based on data rather than through an engineer developing code.<\/p>\n\n\n\n

The\n challenge of focusing on the outcomes is that these need to be defined \nquantitatively. The surprising fact that I have experienced at company \nafter company is that there\u2019s very little alignment on the relevant \nfactors and their relative priority in teams, departments and \norganizations. Although I\u2019ve raised this concern in an earlier blog post on \u201cthe illusion of alignment\u201d<\/a>,\n the problem isn\u2019t just that teams need to agree on what to optimize \nfor, but that any use of machine and deep learning requires a carefully \ndefined set of quantitative success criteria. You can\u2019t ask a system to \ntrain itself without defining what success looks like.<\/p>\n\n\n\n

Although\n many think that the challenge is the machine learning, in many cases \nit\u2019s the selection of the features to measure, track and optimize for \nthat\u2019s the real challenge. This concerns both the input features as well\n as the output features that we\u2019re looking to affect positively. In the \ncase of feature selection, there are two aspects that are challenging.<\/p>\n\n\n\n

The\n first is, in many cases, it\u2019s actually unknown what features are \nrelevant for accomplishing the desired outcome. In this case, the team \nneeds to engage in experimentation in order to gain reliable, \nquantitative insights into the causal relationships between input \nfeatures and desired outcomes. Although this may seem a trivial \nexercise, in many situations there are confounding factors that \ninfluence the causalities and that make it very difficult to distinguish\n between correlation and causation.<\/p>\n\n\n\n

The second is, as I\u2019ve \ndiscussed in an earlier post, the alignment within the team about the \ndesired outcome. Even though different team members may have various \nopinions, again here it may well be the case that the most optimal \noutcome function \u2013 in terms of features and their relative priority \u2013 is\n genuinely unknown. In this case, the team has to start with a first \nguess and then iterate through different models in order to figure out \nthe best overall mix.<\/p>\n\n\n\n

Concluding, as companies are increasingly becoming data- and AI-driven, there\u2019s a growing need to define desired outcomes in quantitative terms. This is the only way that the organization will be able to exploit the benefits provided by digital technologies in general and machine and deep learning in particular. It requires a change in the organization far beyond the data science team: it requires everyone in leadership and decision-making roles to take a more quantitative approach to their job. One can no longer leave this to subordinates; you\u2019ll be required to take this on yourself as well. Or, in short, quantify yourself!<\/p>\n\n\n\n

To get more insights earlier, sign up for my newsletter at<\/em>jan@janbosch.com<\/em><\/a> or follow me on<\/em> janbosch.com\/blog<\/em><\/a>, LinkedIn (<\/em>linkedin.com\/in\/janbosch<\/em><\/a>) or Twitter (<\/em>@JanBosch<\/em><\/a>).<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

Having spent quite a bit of this summer thinking about machine learning and artificial intelligence, it seems to me that there\u2019s a very important transformation ongoing from a focus on the qualitative to a focus on the quantitative. The moment we start with A\/B testing, deploying multi-armed bandits or training machine learning models, the very … Read more<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"generate_page_header":"","footnotes":""},"categories":[15,8,10],"tags":[],"_links":{"self":[{"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/946"}],"collection":[{"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=946"}],"version-history":[{"count":1,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/946\/revisions"}],"predecessor-version":[{"id":948,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/946\/revisions\/948"}],"wp:attachment":[{"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=946"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=946"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=946"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}