Comparing tuned and untuned models

We can compare the best model that we got while tuning the parameters with the best model that we have been using without the help of tuning n_estimators with a value of 50, max_depth with a value of 16, and max_features as auto, and in both the cases it was random forest. The following code shows the values of the parameters of both the tuned and untuned models:

## Random Forests
RF_model1 = RandomForestRegressor(n_estimators=50, max_depth=16, random_state=123, n_jobs=-1)
RF_model1.fit(X_train, y_train)
RF_model1_test_mse = mean_squared_error(y_pred=RF_model1.predict(X_test), y_true=y_test)

## Random Forest with tunned parameters 
RF_tunned_test_mse = mean_squared_error(y_pred=RF_classifier.predict(X_test), y_true=y_test)

To actually see the comparison between the tuned and untuned models, we can see the value of the mean-square error. The following screenshot shows the code to get the mean_squared_error value for the two models, and the plot showing the comparison of the MSE value of the two models:

We can clearly observe here that, on comparison, these tuned parameters perform better than the untuned parameters in both of the random forest models.

To see the actual difference in values between the two models, we can do a little calculation. The following screenshot shows the calculation for getting the percentage of improvement and output showing the actual percentage value:

Here we got an improvement of 4.6% for the tuned model over the untuned one and this is actually very good. In these models, an improvement of 1%-3% percent can also have huge practical implications.