Before running the model, we first have to determine the elements that we will use in building a multilayer perceptron model, shown as follows:
- Architecture: The model contains 23 elements in the input layer, hence we have 25 features in this dataset. We have only one element in the output layer and we will use three hidden layers, although we could use any number of hidden layers. We will use 256 neurons for the first layer, 128 for the second, and 64 for the third one. These are the powers of two.
- Activation function: We will choose the ReLu activation function.
- Optimizing algorithm: The optimization algorithm used here is the Adam optimizer. The Adam optimizer is one of the most popular optimizers as it is the best option for a lot of problems.
- Loss function: We will use the mean squared error because we are doing a regression problem here and this is one of the optimal choices for the loss function.
- Weights initialization strategy: For this, we will use the Xavier initializer, which comes as the default that with the fully_connected function from TensorFlow.
- Regularization strategy: We are not going to use any regularization strategy.
- Training strategy: We are going to use 40 epochs. We will present the dataset 40 times to the network and, in every iteration, we will use batches of 50 data points each time we run the training operation. So, we will use 50 elements of the dataset.