Understanding Transfer Learning in Deep Neural Networks

In this article we understand the transfer learning in deep neural networks

A noteworthy characteristic shared by numerous deep neural networks constructed from photos is their capacity to identify elements in the initial layers such as colors, edges, fluctuations in intensity, and other properties. These characteristics are neither task- or dataset-specific. Which type of picture we use to identify vehicles or lions doesn't matter. In both situations, these low-level characteristics need to be found. Whether the image data or cost function is accurate, these traits still exist. These characteristics can be picked up in a single job, like lion detection. They are also useful for human detection. This is exactly transfer learning.

Finding someone who trains whole convolutional neural networks from scratch is difficult these days. Rather, pre-trained models—like ImageNet, which has 1.2 million photos divided into 1000 categories—are frequently used. These models have been trained using a range of images for tasks that are comparable to one another, and the characteristics are then utilized to tackle new tasks. Transfer learning is characterized by the freezing of layers. "Frozen Layer" refers to a layer that cannot be trained. It may be a hidden layer or a CNN layer. Layers that are not frozen are trained on a regular basis. The weights of the frozen layers will not be updated during training.

Using information from a trained model to solve issues is known as transfer learning. Utilizing the pre-trained model's information may be done in two ways. The pre-trained model's layers are initially frozen, and then layers are trained using our newly acquired dataset. The second method involves starting from scratch and deleting some characteristics from the pre-trained model's layer. A new model can then incorporate these characteristics. In both scenarios, a portion of the previously learnt characteristics are eliminated and the remaining ones are trained. This guarantees that every job uses a single feature. Next, the remaining parts of the model may be trained to adjust to the updated dataset.

It may be unclear how to choose which layers are suitable for training and which to freeze. It is clear that in order to inherit characteristics from a pre-trained model, layers must be frozen. If the model that identified certain blooms isn't working, we need to discover new species. Many of the model's properties will be present in a fresh dataset containing new species. Thus, to maximize the knowledge of that model, we retain fewer layers.

Let us examine all cases in which the target task size and data set are not the same as those of the base network.

1. Compared to the base network data, the target dataset is smaller: We can utilize this target dataset to fine-tune our pre-trained network because it is so little. Overfitting might result from this. The number of classes for the intended job could also vary. In certain situations, it could be necessary to add a new, completely linked layer and delete any older, partially connected ones. We now train only the newly added layers while freezing the remaining portions of our model.

2. Like the base training dataset, the target dataset is sizable: Overfitting won't occur if the dataset is large enough to accommodate a pre-trained model. Here, the final fully linked layer is eliminated and a fresh layer with the appropriate number of classes is introduced. Now, a fresh dataset is used to train the entire model. This preserves the architecture of the model and enables it to be fine-tuned on a sizable new dataset.

3. The target dataset differs from the base network data because it is smaller: Because the target dataset is distinct, high-level feature pre-trained models won't function. A pre-trained model can have the maximum number of layers removed, and new layers added to it to accommodate the number of classes in the new dataset. The remaining layers may then be trained to adapt to a new dataset using the low-level characteristics of the pre-trained model. Even after adding a layer at the end, there are situations where it might be advantageous to train the entire network.

4. Compared to the base network data, the target dataset is larger: It is better to add layers that meet several classes and delete layers from pre-trained networks due to the target network's diversity and complexity. Next, train the network as a whole, keeping no layer frozen.

An issue may be solved quickly and effectively via transfer learning. Transfer learning guides us in the right way. Using this strategy will get the majority of the best results.