Deep learning has been defined as an approach to artificial intelligence in which high-level understanding is obtained through the expression and combination of simpler, low-level representations (Goodfellow et al., 2016).
Today, the use of the term deep learning to describe a machine learning method or algorithm usually implies the following:
- The algorithm is loosely modeled after the human neuron, using artificial neurons. In other words, the building block of a deep neural network is an artificial neuron that takes a sum of weighted input (similar to a regression model) and then applies a nonlinear transformation to that sum.
- Unlike regression, a deep neural network usually has many layers of neurons, with an input layer at the beginning of the model, an output layer at the end of the model, and at least one hidden layer in between the input and output layers. The output of one layer is then fed into the next layer until the output layer is reached.
- To train the weights of the model to predict the output correctly, since there are so many, we must use many, many examples (billions in some cases). A high number of training examples is implied in deep learning. The deeper and more complex the network, the more training examples we need.
- Deep learning uses the backpropagation algorithm to train the weights properly, and the backpropagation algorithm relies on calculus and linear algebra.