{"id":1019,"date":"2020-01-23T16:39:15","date_gmt":"2020-01-23T16:39:15","guid":{"rendered":"http:\/\/janbosch.com\/blog\/?p=1019"},"modified":"2020-01-25T18:52:28","modified_gmt":"2020-01-25T18:52:28","slug":"ai-is-not-big-data-analytics","status":"publish","type":"post","link":"https:\/\/janbosch.com\/blog\/index.php\/2020\/01\/23\/ai-is-not-big-data-analytics\/","title":{"rendered":"AI is NOT big data analytics"},"content":{"rendered":"\n
\"\"
Photo by Robynne Hu on Unsplash\n<\/figcaption><\/figure>\n\n\n\n

During the big data era, one of the key tenets of successfully realizing your big data strategy was to create a central data warehouse or data lake where all data was stored. The data analysts could then run their analyses to their hearts\u2019 content and find relevant correlations, outliers, predictive patterns and the like. In this scenario, everyone contributes their data to the data lake, after which a central data science department uses it to provide, typically executive, decision support (Figure 1).<\/p>\n\n\n\n

\"\"\/
Figure\n 1: Everyone contributes their data to the data lake, after which a \ncentral data science department uses it to provide, typically executive,\n decision support.<\/figcaption><\/figure><\/div>\n\n\n\n

Although this looks great in theory, the reality in many companies \nis, of course, quite a bit different. We see at least four challenges. \nFirst, analyzing data from products and customers in the field often \nrequires significant domain knowledge that data scientists in a central \ndepartment typically lack. This easily results in incorrect \ninterpretations of data and, consequently, inaccurate results.<\/p>\n\n\n\n

Second, different departments and groups that collect data often do \nso in different ways, resulting in similarly looking data but with \ndifferent semantics. These can be minor differences, such as the \nfrequency of data generation, eg seconds, minutes, hours or days, but \nalso much larger differences, such as data concerning individual \nproducts in the field vs similar data concerning an entire product \nfamily in a specific category. As data scientists in a central \ndepartment often seek to relate data from different sources, this easily\n causes incorrect conclusions to be drawn.<\/p>\n\n\n\n

Third, especially with the increased adoption of DevOps, even the \nsame source will, over time, generate different data. As the software \nevolves, the way data is generated typically changes with it, leading to\n similar challenges as outlined above. The result is that the promise of\n the big data era doesn\u2019t always pan out in companies and almost never \nto the full extent that was expected at the start of the project.<\/p>\n\n\n\n

Finally, to gain value from big data analytics requires a strong data\n science skillset and there simply aren\u2019t that many people around that \nhave this skillset. Training your existing staff to become proficient in\n data science skills is quite challenging and most certainly harder than\n providing machine learning education to engineers and developers.<\/p>\n\n\n\n

Many in the industry believe that artificial intelligence \napplications, and especially machine and deep-learning models, suffer \nfrom the same challenges. However, even though both data analytics and \nML\/DL models are heavily based on data, the main difference is that for \nML\/DL, there\u2019s no need to create a centralized data warehouse. Instead, \nevery team, business unit or product organization can start with AI \nwithout any elaborate coordination with the rest of the company.<\/p>\n\n\n\n

\"\"\/
Figure\n 2: Each business unit can build its own ML\/DL models and deploy these \nin the system or solution for which they\u2019re responsible.<\/figcaption><\/figure><\/div>\n\n\n\n

Each business unit can build its own ML\/DL models and deploy these in\n the system or solution for which they\u2019re responsible (Figure 2). The \ndata can come from the data lake or from the local data storage \nsolutions, so you don\u2019t even need to have adopted the centralized data \nstorage approach before starting with using ML\/DL.<\/p>\n\n\n\n

Concluding, AI is not<\/em> data analytics and doesn\u2019t require the same preconditions. Instead, you can start today, just using the data that you have available, even if you and your team are just working on a single function or subsystem. Artificial intelligence and especially deep learning offer amazing potential for reducing cost, as well as for creating new business opportunities. It\u2019s the most exciting technology that has reached maturity in perhaps decades. Rather than waiting for the rest of the world to overtake you, start using AI and DL today.<\/p>\n\n\n\n

This article was inspired by a discussion with Luka Crnkovic-Friis, CEO of <\/em>Peltarion<\/em><\/a>.<\/em><\/p>\n\n\n\n

To get more insights earlier, sign up for my newsletter at <\/em>jan@janbosch.com<\/em><\/a> or follow me on<\/em> janbosch.com\/blog<\/em><\/a>, LinkedIn (<\/em>linkedin.com\/in\/janbosch<\/em><\/a>) or Twitter (<\/em>@JanBosch<\/em><\/a>).<\/em> <\/p>\n","protected":false},"excerpt":{"rendered":"

During the big data era, one of the key tenets of successfully realizing your big data strategy was to create a central data warehouse or data lake where all data was stored. The data analysts could then run their analyses to their hearts\u2019 content and find relevant correlations, outliers, predictive patterns and the like. In … Read more<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"generate_page_header":"","footnotes":""},"categories":[15,4,10],"tags":[],"_links":{"self":[{"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1019"}],"collection":[{"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=1019"}],"version-history":[{"count":3,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1019\/revisions"}],"predecessor-version":[{"id":1025,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/1019\/revisions\/1025"}],"wp:attachment":[{"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=1019"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=1019"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/janbosch.com\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=1019"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}