Data Science – How is the Reality Different from the Expectations?

A few years ago, the extent to which people got curious to know about data science needs no special mention. The tag of being called the most sought-after job of the century did all the wonders. The magic that this tag had on the budding career aspirants cannot be merely put into words. Organizations taking in all possible efforts to unlock transformative growth through AI, ML and data analytics served to be a blessing for those who were all set to make in the mark in the field of data science. What followed is an insane demand for data science as a career path across the globe reached heights. Going by the principle of demand and supply, the high demand eventually paved the way for anticipation that the salaries would be pushed through the roof.

If the reports and numbers pertaining to the data science jobs are to be believed then this is how the situation was like in India – in 2020, the number of data science jobs were expected to see approximately 1.5 lakh new openings. Talking about the salary, data science professionals with experience anywhere between 3 to 10 years bagged a salary that went as high as 6.5 million. Now, this is where we need a clear picture. Following the same lines as that of most technology trends, the hype and the reality of what data science can accomplish aren't aligned. Let's find out.

Data science – How is the reality different from the expectations?

The pandemic has had an intense impact on almost everything that one can think of. The technological world, too, saw drastic changes as a result of the pandemic. The organizations had started investing heavily in advanced analytics and next-generation technologies like IoT, blockchain and quantum computing, expecting better results. As a matter of fact, not all organizations were able to make the best out of their investment. The reality – for most organizations, their investments in AI and machine learning have failed to produce the promised results. To make things worse, the mismatch between the demand and supply of data science professionals has always been an area of concern. Who is to be blamed for this?

Well, the data scientists are not to be blamed here. They have done their part – undergone training. So, what's the problem then? Right from the shortage of talent to the gap between data science talent, the AI and machine learning technology that the data scientists are trained to use, and the business problems they aim at solving, all this has resulted in preventing the promised value from being delivered.

Over the last few years, almost every company has eyed joining the AI race. In that process, they are creating their own data, science teams. What they have overlooked in this process is due diligence before hiring data science professionals or developing the practice in-house. Having a clearly laid strategy and vision pertaining to how their AI investment is going to play out, in the long run, is what most of the organizations didn't pay heed to. In the long run, when these organizations realize that there isn't any tangible value from large data science teams – data science professionals are the first to get laid off.

Though the trained data science professionals have vast knowledge with regard to computer science, mathematics, and statistical models, it has been observed that they lack business applications and domain expertise. It is high time that we acknowledge the reality – overemphasis on technical skills and under-emphasis of business and soft skills has led to not only a communication gap but an imagination gap as well.

Getting things straight – data science is here to stay and the demand for data science professionals is bound to be there. Nurturing the right talent, data scientists' up-skilling themselves and putting the AI investment in the right place is the need of the hour