Training the decision tree classifier

Let's learn how to train the decision tree classifier as shown in the following code snippet:

In []: 
from sklearn import tree 
tree_model = tree.DecisionTreeClassifier(criterion='entropy', random_state=42) 
tree_model = tree_model.fit(X_train, y_train) 
tree_model 
Out[]: 
DecisionTreeClassifier(class_weight=None,  
            criterion='entropy', max_depth=None, 
            max_features=None, max_leaf_nodes=None, 
            min_impurity_split=1e-07, min_samples_leaf=1, 
            min_samples_split=2, min_weight_fraction_leaf=0.0, 
            presort=False, random_state=42, splitter='best') 

The most interesting for us are the class attributes of DecisionTreeClassifier:

These attributes are known as hyperparameters. They are different from model parameters: the former is something that users can tweak, and the latter is something that machine learning algorithm learns. In a decision tree, parameters are specific rules in its nodes. The tree hyperparameters must be adjusted depending on the input data, and this is usually done using cross-validation (stay tuned).

Decision tree classifier documentation: http://scikit-learn.org/stable/modules/tree.html.

The properties of the model, which are not adjusted (learned) by the model itself, but are available for the user's adjustments, are known as hyperparameters. In the case of the decision tree model, these hyperparameters are class_weight, criterion, max_depth, max_features, and so on. They are like knobs you can turn to adjust the model to your specific needs.