There are three typical ways of using pre-trained neural networks. They are listed as follows:
- Reusing the models: Consider an example where we apply a pre-trained neural network and retrieve the image class. Depending on performance requirements, this could be MobileNet, Inception, VGG, ResNet, or any other that is available out there.
- Changing the network structure: Consider an example where we are replacing the last FullyConnected and SoftmaxOutput layers with a different combination. This would be similar to using a CIFAR-10 model with an MNIST dataset. The difference in the implementation is very tiny and updating the last layers would boost the training time.
- Feature extraction: Use a pre-trained network as a feature extractor by using the last layer activations as features. These features can be an input to other neural networks.
Working with a pre-trained network has a number of benefits in comparison to building a new network from scratch, which are enlisted here:
- No need to develop custom architecture
- Very easy to deploy
- Faster training time
- Higher accuracy
First of all, it makes the process of learning much faster and more comfortable than training from scratch. You do not need to work and spend time on deciding the network architecture as the one you will be reusing is already well-tested. Usually, pre-trained networks do not have any special requirements; therefore, it makes the models easy to deploy. Last but not least, there is a higher chance of getting a better score using the pre-trained network as a base.