Part III of this book has been a build-up to a revolution. In this last chapter, we’ll finally examine the breakthrough known as deep learning.
With an additional car chase or two, the story of deep learning would make a great Hollywood movie. You know its beginning from Chapter 8, The Perceptron: the clash between connectionists and symbolists, and how the connectionists were discredited and pushed into the basements of research.
You might remember the follow-up to that story from Part II of this book. In the 1970s, ML researchers discovered how to train neural networks with backpropagation. After years of struggle, they could finally overcome the limitations of perceptrons and tackle nonlinear data! For a while, it looked like connectionism was back—but as it happened, that new spark was also bound to fizzle out.
This chapter continues the story, up to the birth of deep learning. It also explains, to the best of what we know today, what makes deep learning so effective and such an improvement upon earlier methods. Finally, the last pages will point at a few trails you can walk after closing this book to delve deeper into this brave new world.