Bidirectional with 1D convolutional neural network model

So far, the best approaches we have seen are from the 1D convolutional neural network model which got 95.73%, and the gated recurrent units model which got 95.90% accuracy. The following code combines them! The code for our bidirectional with 1D convolutional neural network model is in Chapter7/classify_keras8.R.

The parameters for the model are max length=150, the size of the embedding layer=32, and the model was trained for 10 epochs:

word_index <- dataset_reuters_word_index()
max_features <- length(word_index)
maxlen <- 250
skip_top = 0

..................

model <- keras_model_sequential() %>%
  layer_embedding(input_dim = max_features, output_dim = 32,input_length = maxlen) %>%
  layer_spatial_dropout_1d(rate = 0.25) %>%
  layer_conv_1d(64,3, activation = "relu") %>%
  layer_max_pooling_1d() %>%
  bidirectional(layer_gru(units=64,dropout=0.2)) %>%
  layer_dense(units = 1, activation = "sigmoid")

..................
  
history <- model %>% fit(
  x_train, y_train,
  epochs = 10,
  batch_size = 32,
  validation_split = 0.2
)

Here is the output from the model's training:

Train on 8982 samples, validate on 2246 samples
Epoch 1/10
8982/8982 [==============================] - 26s 3ms/step - loss: 0.2891 - acc: 0.8952 - val_loss: 0.2226 - val_acc: 0.9319
Epoch 2/10
8982/8982 [==============================] - 25s 3ms/step - loss: 0.1712 - acc: 0.9505 - val_loss: 0.1601 - val_acc: 0.9586
Epoch 3/10
8982/8982 [==============================] - 26s 3ms/step - loss: 0.1651 - acc: 0.9548 - val_loss: 0.1639 - val_acc: 0.9541
Epoch 4/10
8982/8982 [==============================] - 26s 3ms/step - loss: 0.1466 - acc: 0.9582 - val_loss: 0.1699 - val_acc: 0.9550
Epoch 5/10
8982/8982 [==============================] - 26s 3ms/step - loss: 0.1391 - acc: 0.9606 - val_loss: 0.1520 - val_acc: 0.9586
Epoch 6/10
8982/8982 [==============================] - 26s 3ms/step - loss: 0.1347 - acc: 0.9626 - val_loss: 0.1626 - val_acc: 0.9550
Epoch 7/10
8982/8982 [==============================] - 27s 3ms/step - loss: 0.1332 - acc: 0.9638 - val_loss: 0.1572 - val_acc: 0.9604
Epoch 8/10
8982/8982 [==============================] - 26s 3ms/step - loss: 0.1317 - acc: 0.9629 - val_loss: 0.1693 - val_acc: 0.9470
Epoch 9/10
8982/8982 [==============================] - 26s 3ms/step - loss: 0.1259 - acc: 0.9654 - val_loss: 0.1531 - val_acc: 0.9599
Epoch 10/10
8982/8982 [==============================] - 28s 3ms/step - loss: 0.1233 - acc: 0.9665 - val_loss: 0.1653 - val_acc: 0.9573

The best validation accuracy was after epoch 6, when we got 96.04% accuracy, which beats all of the previous models.