CART trees were introduced in 1984 and the algorithm is characterized by the display of a series of questions and answers to determine what the next question will be. The main components of these trees are:\u00a0<\/p>\n\n\n\n
- The rules for determining the division of data for each node<\/li>
- The rules to determine where a branch ends<\/li>
- Prediction of the objective value in each terminal node<\/li><\/ol>\n\n\n\n
The main advantages of CART trees are:<\/p>\n\n\n\n
- It is a non-parametric model so it does not depend on the type of distribution to which the data belongs.<\/li>
- They are not directly impacted by out-of-range values \u200b\u200bor outliers.<\/li>
- It incorporates both training and test data and a cross validation to evaluate the goodness of fit<\/li><\/ul>\n\n\n\n
With all this, a decision tree is a graphic representation of all possible solutions to help make a decision. The basic intuition behind this type of tree is to divide a large data set into smaller subsets under certain rules until you get a set of data small enough to establish a simple label.<\/p>\n\n\n\n
Each characteristic that allows to make the division is denoted by a node of the tree while the branches represent the possible decisions. The result of the decision is indicated in a leaf node without branches.<\/p>\n\n\n\n
![\"\"](\"https:\/\/www.jacobsoft.com.mx\/wp-content\/uploads\/2018\/10\/decision_tree_example.png\")
<\/figure><\/div>\n\n\n\nIn the previous image we see a decision tree model where we negotiate with a binary classification problem, although in many cases we can have decision trees for multiple classes. In the previous example we have only two decision options, accept or decline the job offer. The branch to select is the one that provides us with the most information to reduce the degree of randomness in our decision.\u00a0<\/p>\n\n\n\n
Finally, the division of data is done with the intention of minimizing entropy and maximizing data groups.<\/p>\n\n\n\n
Decision trees with Python<\/h2>\n\n\n\n
For the example, we use a data file with information about customers who have purchased or not online. If I buy the value of the dependent variable is 1 and if I do not buy, the value is 0. The data in the independent variable are gender, age and estimated salary.<\/p>\n\n\n\n
![\"\"](\"https:\/\/www.jacobsoft.com.mx\/wp-content\/uploads\/2018\/10\/Compras_en_linea.png\")
<\/figure>\n\n\n\nWe will use the age and the estimated salary to create a classification tree with the labels of bought (1) or not bought (0) and thus be able to classify new records. For this we create the tree with a set of data that we call the training set.<\/p>\n\n\n\n\n
# Clasification with Decision Trees<\/span>\n\n# Import of libraries<\/span>\nimport<\/span> numpy ace<\/span> np\nimport<\/span> matplotlib.pyplot ace<\/span> plt\nimport<\/span> pandas ace<\/span> P.S\n\n# Importation of the dataset<\/span>\ndataset = pd.read_csv ('Compras_en_Linea.csv<\/span>') X = dataset.iloc [:, [2<\/span>, 3<\/span>]]. values \u200b\u200by = dataset.iloc [:, 4<\/span>] .values\n\n# Division of data set in training data # and test data<\/span>\ndesde<\/span> sklearn.cross_validation import<\/span> train_test_split X_train, X_test, y_train, y_test = train_test_split (X, y, test_size = 0.25<\/span>, random_state = 0<\/span>)\n\n# Scale adjustment<\/span>\ndesde<\/span> sklearn.preprocessing import<\/span> StandardScaler sc = StandardScaler () X_train = sc.fit_transform (X_train) X_test = sc.transform (X_test)\n\n\n\n# We create the Decision Tree for Classification and train it<\/span>\ndesde<\/span> sklearn.tree import<\/span> DecisionTreeClassifier classifier = DecisionTreeClassifier (criterion = 'entropy<\/span>', random_state = 0<\/span>) classifier.fit (X_train, y_train)\n<\/pre>\n\n\n\nWhen executing this piece of code we are loading the data set with 400 customer records of which 25% will be used to test the model and 75% (300 records) for training or to create the tree under the criteria of the age and the estimated salary (columns 2 and 3).<\/p>\n\n\n\n\n
\nimport<\/span> pydotplus\ndesde<\/span> sklearn.tree \nimport<\/span> export_graphviz dot_data = export_graphviz (classifier, out_file = None, filled = True, feature_names = ['Age', 'Salary']) graph = pydotplus.graph_from_dot_data (dot_data) graph.write_pdf ('tree.pdf')\n<\/pre>\n\n\n\n![\"\"](\"https:\/\/www.jacobsoft.com.mx\/wp-content\/uploads\/2018\/10\/arbol_sin_escalas.png\")
<\/figure><\/div>\n\n\n\nGiven the size of the training data set, the tree is too large, but when viewing the image in detail, we observe that each leaf or end node has an entropy value equal to zero. In the same node shows us the number of records that meet the criteria.<\/p>\n\n\n\n
If we now adjust the tree for the 100 records in the test set, the resulting graph is smaller and can be seen better.<\/p>\n\n\n\n
![\"\"](\"https:\/\/www.jacobsoft.com.mx\/wp-content\/uploads\/2018\/10\/arbol_test.png\")
<\/figure><\/div>\n\n\n\nIn the final node located on the right side at the third level, it says entropy = 0, samples = 13 and value = [0, 13] is a blue node that indicates that clients older than 42.5 and salary greater than 84.5 if they buy.<\/p>\n\n\n\n
Now if we make the prediction for the test set and check the confusion matrix, we have:<\/p>\n\n\n\n\n
# Predicting the Test set results<\/span>\ny_pred = classifier.predict (X_test)\n\n# Making the Confusion Matrix<\/span>\ndesde<\/span> sklearn.metrics import<\/span> confusion_matrix cm = confusion_matrix (y_test, y_pred)\n<\/pre>\n\n\n\n![\"\"](\"https:\/\/www.jacobsoft.com.mx\/wp-content\/uploads\/2018\/10\/arbol_decision_matriz_confusion.png\")
<\/figure><\/div>\n\n\n\nWe observe that of 100 records of the set of tests, there are 9 errors, that is, 3 elements that should have been classified with 1 were classified with 0, these represent the false positives. 6 records were classified with 1 and should be classified with 0, the latter represent false negatives.\u00a0<\/p>\n\n\n\n
This implies that the accuracy of the model is 0.91, that is, 91% much better than that which resulted from the same set of tests, Logistic regression<\/a> of the previous article.<\/p>\n\n\n\n
![\"\"](\"https:\/\/www.jacobsoft.com.mx\/wp-content\/uploads\/2018\/10\/decision_tree_graph.png\")
<\/figure><\/div>\n\n\n\nThe previous graph is created for the set of tests and in it we see the decision boundaries for the data, the green zone is for customers who do buy and the red zone for those who do not.<\/p>\n