Understanding decision trees

Im new to decision trees and am trying some decision tree modelling now with titanic data.

I have the following dataset.

 train <- read.csv(url("http://s3.amazonaws.com/assets.datacamp.com/course/Kaggle/train.csv"))


Now when I create a decision tree doing this:

my_tree_two <- rpart(Survived ~ Sex + Age, data=train, method="class")
my_tree_two


I get the following output.

n= 891 node), split, n, loss, yval, (yprob)
* denotes terminal node

1) root 891 342 0 (0.6161616 0.3838384)
2) Sex=male 577 109 0 (0.8110919 0.1889081)
4) Age>=6.5 553  93 0 (0.8318264 0.1681736) *
5) Age< 6.5 24   8 1 (0.3333333 0.6666667) *
3) Sex=female 314  81 1 (0.2579618 0.7420382) *


I understand most of it but what I do not understand is why age divided into age >= 6.5 and < 6.5? Could anybody elaborate on this?

• The algorithm always puts splits at the midpoint of two data values. Aug 17, 2015 at 17:13
• A great tutorial of the Titanic data in R was created by Trevor Stephens. Decision tree page is here Aug 17, 2015 at 17:15
• @MatthewDrury thanks for your input. So this is the top and the bottom of the intervals for Male en Female divided by two? Aug 17, 2015 at 17:38

Apparently the goal is to split your observations in two classes, the class is given by the binary variable 'yval'

You 'predict' yval with a decision rule. Your decision rule is:

if ( sex == "male" ) {
# there are 577 males in your dataset
if ( Age >= 6.5)               # amongst the 577 males you have 553 with age >= 6.5
yval <- 0 # so the class is '0'
else   #Age < 6.5              # amongst the 577 males you have 24 with age < 6.5
yval<- 1
}
else  # sex is female
# there are 314 females in your dataset
yval <- 1.


Obviously, the class predicted by your decision rule can be erroneous and different from the class that was observed in your dataset.

In the leaf node 'sex == female' there are 25.8% of your cases that are class 0 and 74.2% that are 1, as, see the above decision rule, you decide to put them all in class 1 you make an error for 25.8% of all females.

The underlying algorithm in rpart that divides the tree into branches. The algorithm tries to make each leaf node 'as pure as possible', i.e. with as much possible cases of the same observed class in the leaf node. Therefore these algorithms use an 'impurity measure', there exists more than one impurity measure, for details see section 9.2.3 of this pdf

• @f coppens, thanks for your answer. Could you also tell me why r divides Age into >=6.5 and smaller <6.5 Aug 17, 2015 at 19:45
• @Marc van der Peet: it is the algorithm implemented in rpart that divides the tree into branches, not R. The algorithm tries to make each leaf node 'as pure as possible', i.e. with as much possible cases of the same observed class in the leaf node. Therefore these algorithms use an 'impurity measure', there exists more than one impurity measure. Details are in section 9.2.3 of web.stanford.edu/~hastie/local.ftp/Springer/OLD/…
– user83346
Aug 17, 2015 at 19:52
• @MarcvanderPeet Is the confusion that 6.5 is not a true data value, i.e. not a valid value of Age? Aug 19, 2015 at 14:40