Data-Centric Practices To Stir Up AI Engineering! Is AI Democratization Within Reach?

A data-centric approach entails focussing more on data quality tools to work around noisy data

One common thing every machine learning or artificial intelligence project always is lack adequate quality data – data that is consistent, accurate, and relevant. Embark on developing a novel machine learning model, the very input sources, and datasets, become impediments. The comprehensive performance of an AI model depends on the variability of data sets – which ironically are not available in a data-driven digital world. Blame it on privacy policies that prevent data gathering or inadequacies in the model development, adopting a structural design inefficient to leverage all the data available, it is the model which suffers in the end. As the famous AI trainer and developer Andrew Ng says, "Data is food for artificial intelligence", signifying the importance of moving away from a model-centric approach to a data-centric approach, it is possible to get the best performance out of an ML model by improving the quality of the data.

What is a data-centric approach?

This approach involves altering the quality of datasets to make them suitable for ML model training. Rather than going on a data gathering sojourn, more energy is focused on data quality tools to work around noisy data. When you ask why noisy data is such a hindering factor here, we need to remember, that ML model algorithms are trained to make decisions without knowing how to make them just by identifying a pattern from the data it is trained on. Predictably, the wisest algorithm ever trained, used the largest datasets which included human decisions and transactions. However, if only an additional parameter, called context could be taught, a data-driven approach would have been sufficient for AI engineering. "We could train a language model for Gboard—Android smartphones' predictive keyboard—on, say, Wikipedia data, but it would be terrible because people don't type text messages anything like they write Wikipedia articles," says Brendan McMahan, a senior researcher at Google AI, the company's artificial intelligence branch.

Why MLoPs not enough?

If we look closely at the machine learning cycle, we can understand why most ML projects do not get through the final stage. Usually, an ML cycle has three stages – collecting data, training the model, and model deployment, where analysis of training and deployment results can result in another cycle of data collection and model training. The strategy essentially involves implementing corrections by looking into test data, which apparently will not yield results for untrained circumstances, hence defeating the goal of model robustness. This apart, difficulty in standardizing a workflow and scaling up AI models remains one another area, and the code-centric approach is notorious for being rigid for impending upgrades.

Democratic AI on its able wings:

While we take so much pride in the novelty of AI models, the fact remains that it is still out of reach to many companies. The reasons are many of which accountability of AI models in fulfilling the model objectives remains the primary one. Reliability and accountability can be expected from the model only when the right data is fed to the model, and it forms a part of democratizing data management and thus democratizing AI. But lack of quality data is a perpetual problem in AI modeling and dissemination which a data-centric approach can solve to a great extent. Therefore, it is essential to get out of the black box model, designing the system with the data that the model requires for a wider application.