{"id":1167,"date":"2020-11-24T13:03:00","date_gmt":"2020-11-24T13:03:00","guid":{"rendered":"http:\/\/janbosch.com\/blog\/?p=1167"},"modified":"2020-11-24T13:03:02","modified_gmt":"2020-11-24T13:03:02","slug":"towards-autonomously-improving-systems","status":"publish","type":"post","link":"https:\/\/janbosch.com\/blog\/index.php\/2020\/11\/24\/towards-autonomously-improving-systems\/","title":{"rendered":"Towards autonomously improving systems"},"content":{"rendered":"\n
\"\"
Image by Mediamodifier from Pixabay<\/figcaption><\/figure>\n\n\n\n

This week, I attended the International Conference on Software Business (ICSOB 2020<\/a>) and gave a presentation on autonomously improving systems. The core idea is that software-intensive systems can measure their performance, know what to optimize for and can autonomously experiment with their own behavior.<\/p>\n\n\n\n

The history of software-intensive systems can be divided into three \nmain phases. In the first phase, we built systems according to a \nspecification. This specification could either come from the customer or\n from product managers, but the goal of the R&D team was to build a \nsystem that realized the requirements in the specification.<\/p>\n\n\n\n

\"\"\/
Evolution of software-intensive systems<\/figcaption><\/figure>\n\n\n\n

Many online companies, but also the first embedded-systems companies,\n are operating in the next stage. Here, teams do not get a specification\n but rather a set of one or more prioritized KPIs to improve. In \ne-commerce, the predominant KPI often is conversion; in embedded \nsystems, often a weighted mix of performance, reliability and other \nquality attributes is used. Teams get as a target to improve one or more\n of these KPIs without (significantly) deteriorating the others. They \nhave to develop hypotheses on this and test them in the field using, for\n instance, A\/B experiments.<\/p>\n\n\n\n

Although the second stage is a major step forward for many companies,\n the problem is that it still is the team doing all the heavy lifting. \nEspecially running many A\/B experiments can be quite effort consuming. \nThe next step that some companies are starting to explore is to allow \nthe system to generate its own experiments with the intent to learn \nabout ways to improve its own performance. Theoretically, this falls in \nthe category of reinforcement learning, but it proves to be quite \nchallenging to realize this is an empirical, industrial context. advertorial \n<\/p>\n\n\n\n

The evolution companies go through to reach this third stage can be \nput in a model, showing the activities and technologies that can be used\n at each level. From level 2, we see some autonomously improving system \nbehavior such as adding intelligent, online selection of experiments, as\n well as automatically generating experiments. This results in all kinds\n of challenges, including predicting the worst-case performance of the \ngenerated alternatives. If a system autonomously generates and deploys \nexperiments, some of these experiments can exhibit very poor \nperformance, meaning the system requires models to predict the \nworst-case outcome for each experiment, as well as solutions to cancel \nongoing experiments if performance is negatively affected.<\/p>\n\n\n\n

\"\"\/
Evolution model<\/figcaption><\/figure>\n\n\n\n

With the increasing prevalence of AI, we need to start looking into online reinforcement learning in software-intensive systems as this would facilitate autonomously improving systems. This ambition comes with major challenges that we\u2019re now researching. However, I encourage you to start exploring where the systems that you build could autonomously improve their own behavior. Even starting in a small, risk-free corner of the system can be very helpful to learn about this paradigm. The overall goal is that every day I use your product, I want it to be better!<\/p>\n\n\n\n

To get more insights earlier, sign up for my newsletter at\u00a0<\/em>jan@janbosch.com<\/em><\/a> or follow me on<\/em> janbosch.com\/blog<\/em><\/a>, LinkedIn (<\/em>linkedin.com\/in\/janbosch<\/em><\/a>), Medium (<\/em>https:\/\/janbosch.medium.com\/) or Twitter (<\/em>@JanBosch<\/em><\/a>).<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

This week, I attended the International Conference on Software Business (ICSOB 2020) and gave a presentation on autonomously improving systems. The core idea is that software-intensive systems can measure their performance, know what to optimize for and can autonomously experiment with their own behavior. The history of software-intensive systems can be divided into three main … Read more<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"generate_page_header":"","footnotes":""},"categories":[15,9,10],"tags":[],"_links":{"self":[{"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1167"}],"collection":[{"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=1167"}],"version-history":[{"count":1,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1167\/revisions"}],"predecessor-version":[{"id":1169,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1167\/revisions\/1169"}],"wp:attachment":[{"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=1167"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=1167"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=1167"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}