In Fearful Symmetry, we learned that we shouldn’t initialize all the neural network’s weights to the same value. See for youself what happens if we ignore that advice.
You can use NumPy’s zeros function to initialize all the weights to 0. For example, here is how you get a matrix of zeros with two rows and three columns:
| => | np.zeros((2, 3)) |
| <= | array([[ 0., 0., 0.], |
| | [ 0., 0., 0.]]) |
The parameter to zeros has its own parentheses, because it’s a tuple—an immutable collection of values. (For more about tuples, read Collections.) In this case, the tuple has two values, that are the rows and columns of the matrix, respectively.
After you initialize the weights to zero, run the network for a few iterations, and see what happens. Compare its results to the results of the randomly initialized network. How does it fare? Also try modifying the zero-initialized network to give it fewer hidden nodes. Do you see a significant difference in accuracy?
Finally, peek into the zero-initialized network as it trains. For example, here is how you can print ten rows from around the middle of w₁ and w₂:
| | print(w1[120:130]) |
| | print(w2[120:130]) |
What do you see?