In the last few pages, you got acquainted with a new loss function (the cross-entropy loss) and a few new concepts, like numerical stability. The focus of this chapter, however, was on implementing the neural network that we designed in the previous chapter. Here is all the code that we came up with:
| | import numpy as np |
| | |
| | |
| | def sigmoid(z): |
| | return 1 / (1 + np.exp(-z)) |
| | |
| | |
| | def softmax(logits): |
| | exponentials = np.exp(logits) |
| | return exponentials / np.sum(exponentials, axis=1).reshape(-1, 1) |
| | |
| | |
| | def loss(Y, y_hat): |
| | return -np.sum(Y * np.log(y_hat)) / Y.shape[0] |
| | |
| | |
| | def prepend_bias(X): |
| | return np.insert(X, 0, 1, axis=1) |
| | |
| | |
| | def forward(X, w1, w2): |
| | h = sigmoid(np.matmul(prepend_bias(X), w1)) |
| | y_hat = softmax(np.matmul(prepend_bias(h), w2)) |
| | return y_hat |
| | |
| | |
| | def classify(X, w1, w2): |
| | y_hat = forward(X, w1, w2) |
| | labels = np.argmax(y_hat, axis=1) |
| | return labels.reshape(-1, 1) |
| | |
| | |
| | def report(iteration, X_train, Y_train, X_test, Y_test, w1, w2): |
| | y_hat = forward(X_train, w1, w2) |
| | training_loss = loss(Y_train, y_hat) |
| | classifications = classify(X_test, w1, w2) |
| | accuracy = np.average(classifications == Y_test) * 100.0 |
| | print("Iteration: %5d, Loss: %.6f, Accuracy: %.2f%%" % |
| | (iteration, training_loss, accuracy)) |
With that, half of our neural network’s logic is done—the half that takes care of the classification phase.
This code, however, won’t be useful until we’ve also written the code that trains the network. In the next chapter we’ll do that, and we’ll test drive the entire thing.