Deep learning is a set of complex technologies, but they all spring from a simple idea: neural networks become more powerful if they have many layers. Those “deep” networks pose more challenges than the “shallow” three-layered networks that we built so far. In the next few chapters, we’ll describe and overcome those challenges.
Adding layers is just the beginning. In the last few years, the original concept of deep learning branched out into many innovative—and sometimes wonderful—ideas. We’ll take a look at some of those.
In Parts I and II, we used MNIST as our benchmark. MNIST will also be our starting point in the first chapters of Part III—but eventually we’ll outgrow it, and tackle more complex datasets. Brace yourself!