Aspects that will Shape Data Science in the New Normal

The year 2021 has made the application of data science even more central to analyzing business situations.

Digital-first business models have unlocked a wealth of data for enterprises to derive powerful insights from rapidly evolving customer preferences and behaviors. At a time when digitally empowered customers are seeking ever higher levels of personalization, engagement and adaptability, a piecemeal and reactive approach to change can throw enterprises off their game. This is where data presents enterprises with rich new opportunities to innovate and experiment ahead of time.

Pandemic-driven shifts,be it due to lockdown-imposed constraints or consequent consumer patterns,have made the application of data science even more central to analyzing business situations. By continually finding new patterns and correlations, enterprises can refine business models and create new ones, while factoring in the right combination of technologies.This is where the new normal has accentuated certain aspects of data science.

Commoditization of AI/ML choices

Leading tech giants and academia are increasingly contributing to the body of knowledge on data sciences by publishing their models, also referred to as State-of-the-Art (SOTA) models. Organizations are directly using these models as pre-built application programming interfaces (APIs)in their AI solutions.

A case in point is the accent of natural language processing(NLP) on machine reading comprehension. By now, more than 60 NLP models or variations have surpassed human performance. This is based on Stanford Question Answering Dataset (SQuAD 2.0),which is one of the largest reading comprehension datasets that test the ability of a system to read a passage of text and answer questions about it. Given this increased accessibility to pre-built APIs, it would be fair to say that in many situations, the application of AI is more about solving problems than building complex models. The focus is on stitching together a set of narrow intelligences into solutions that can address specific business problems.

A related problem,consequently, is the availability of training data, which can help orchestrate these readymade models for various business scenarios in an organization. Therefore, the focus has shifted to using automated techniques to significantly reduce the time and effort required to prepare labelled data.

Constant experimentation and monitoring

Data science must account for fluid patterns changing with reality. For example,employees opting to work from office on certain days of the week in a hybrid working environment have a bearing on patterns ranging from traffic to visits to physical locations. Such dynamism and uncertainty give rise to the need to experiment with and test new hypotheses all the time, which is not possible without a strong data infrastructure with automated machine learning operations (MLOps)pipelines.

Constant post-deployment monitoring is required in view of potential model drifts because of changing consumer needs. Considering the constant cooperation and exchange of information across devices, the environment also needs to provide for mixed and challenger models. Competition between machine learning models allows the ability to select the most optimal one at runtime.An automated platform-based approach is an absolute must to drive data science for business differentiation.

Solving for the problem of platforms

Platform selection is a vital consideration. Should it be a cloud-native solution or a proprietary platform? Do proprietary platforms respect the SOTA world? Are they cloud scalable? These are among the key questions — made more pronounced by the multi-cloud reality so predominant across most enterprises — that call for answers. Typically, different departments in an enterprise invest in different clouds and AI platforms. As a result, synergizing them, and enforcing common architecture and monitoring frameworks are among the key considerations for enterprise architecture teams.

Addressing the reality of delivery

Applying data science is an important prerequisite for businesses to effectively navigate the new normal. However, in reality, there is no such thing as a data science problem. Instead, what enterprises look to address are business problems. Using data to do so effectively is about mastering problem-farming capabilities and backing them up with a platform-led approach realized through integrated service delivery. Delivering data science services requires a multitude of skills — business problem framers, machine learning engineers, cloud engineers, MLOps specialists, visual and data experts, and so on.Only by building the capabilities and infrastructure to constantly experiment with and monitor data can enterprises harness its full power for competitive advantage.

Author:

Manoj Karanth, Vice President, Data and Intelligence, Mindtree