Understanding the Limitations of Deep Learning Models

While Deep Learning is rapidly gaining popularity across industries, why is it not a mass norm yet?

Deep learning is receiving a lot of hype at the moment. The reasons behind this popularity are availability of huge dataset, recent breakthroughs in the development of algorithms, impressive computational power, and glamourized marketing. However, recently, limitations of deep learning have become a central theme at many artificial intelligence debates and symposiums. Even deep learning pioneer Yoshua Bengio has acknowledged the flaws of this widely used technology.

Deep learning has offered noteworthy capabilities and advances in voice recognition, image comprehension, self-driving car, natural language procession, search engine optimization and more.  Did you know that despite such promising scope of deep learning, this variant of artificial intelligence garnered huge sensation in the third iteration i.e. the 2000s-present. With the emergence of GPUs, deep learning could progress beyond its competition on a plethora of benchmarks and real-world applications. Even the computer vision (one of common use cases of deep learning) community was fairly skeptical until AlexNet demolished all its competitors on ImageNet, in 2011.

Though even after these developments, there are many limitations in deep learning model that hinder its mass adoption today. For instance, the models are not scalable and rotation invariants and can easily misclassify images when the object poses are unusual. Let's focus on some of the common drawbacks.

A major downslide is that deep learning algorithms require massive datasets for training. To exemplify, for a speech recognition program, data formulating multiple dialects, demographics and time scales are required to obtain desired results. While major tech giants like Google and Microsoft are able to gather and have abundant data, small firms with good ideas may not be able to do so. Also, it is quite possible that sometimes, the data necessary for training a model is already sparse or unavailable.

Besides, with larger architecture, the deep learning model gets more data hungry to produce viable results. In such scenarios, reusing data may not be an appropriate idea, and data augmentation could be useful to some extent, but having more data is always the preferred solution. Additionally, training deep learning models is an extremely costly affair due to complex data models. Sometimes, they require expensive GPUs and hundreds of machines, which adds up the cost for the users.

Next, deep learning models that perform well on benchmarked datasets, can fail badly on real world images outside the dataset. To illustrate this, consider a deep learning algorithm which learns that school buses are always yellow, but, all of a sudden, school buses become blue, it will need to be retrained. On contrary, a five-year-old would have no problem recognizing the vehicle as a blue school bus. Moreover, they also fail to perform efficiently in situations that may be little dissimilar to the setting they have trained with. For e.g. Google's DeepMind trained a system to beat 49 Atari games; however, each time the system beat a game, it had to be retrained to beat the next one.

This brings us to another limitation of deep learning i.e. while the model may be exceptionally good at mapping inputs to outputs it may not be good at understanding the context of the data they're handling. In other words, it lacks common sense, to draw conclusions in cross-domain boundary areas. As per Greg Wayne, an AI researcher at DeepMind, current algorithms may fail to discern that sofas and chairs are for sitting. It also falls short of general intelligence and multiple domain integration.

Deep learning algorithms also counter the opacity or black box problem, making them hard to debug or understand how they make decisions. It also leaves users at a loss when it comes to understanding why certain parts fail. Generally, deep learning algorithms sift through millions of data points to find patterns and correlations that often go unnoticed to human experts. While it may be an issue in performing trivial tasks, in situations like tumor detection, the doctor needs to know why the model marked some areas and why it didn't for others in a scanning report.

Further, imperfections in the training phase of deep learning algorithms make them vulnerable to adversarial samples: inputs crafted by adversaries with the intent of causing deep neural networks to misclassify. Meanwhile, the presence of biases in the datasets can lead to inexact outcomes – thus inherently amplifying the discrimination in real world. Existence of black box can make it challenging for the developers to identify where, how such maligned data was fed to the system.

Lastly, deep learning architectures possess excellent abilities, like image classification and predicting a sequence. They can even generate data that matches the pattern of another like GANs. However, they fail to generalize to every supervised learning problem.