Understanding decision trees

Question

Im new to decision trees and am trying some decision tree modelling now with titanic data.

I have the following dataset.

 train <- read.csv(url("http://s3.amazonaws.com/assets.datacamp.com/course/Kaggle/train.csv"))

Now when I create a decision tree doing this:

my_tree_two <- rpart(Survived ~ Sex + Age, data=train, method="class")
my_tree_two

I get the following output.

n= 891 node), split, n, loss, yval, (yprob)
  * denotes terminal node

1) root 891 342 0 (0.6161616 0.3838384)  
2) Sex=male 577 109 0 (0.8110919 0.1889081)  
4) Age>=6.5 553  93 0 (0.8318264 0.1681736) *
5) Age< 6.5 24   8 1 (0.3333333 0.6666667) *
3) Sex=female 314  81 1 (0.2579618 0.7420382) *

I understand most of it but what I do not understand is why age divided into age >= 6.5 and < 6.5? Could anybody elaborate on this?

Answer 1

Apparently the goal is to split your observations in two classes, the class is given by the binary variable 'yval'

You 'predict' yval with a decision rule. Your decision rule is:

if ( sex == "male" ) { 
   # there are 577 males in your dataset
   if ( Age >= 6.5)               # amongst the 577 males you have 553 with age >= 6.5
     yval <- 0 # so the class is '0'
   else   #Age < 6.5              # amongst the 577 males you have 24 with age < 6.5
     yval<- 1
}
else  # sex is female
  # there are 314 females in your dataset
     yval <- 1.  

Obviously, the class predicted by your decision rule can be erroneous and different from the class that was observed in your dataset.

In the leaf node 'sex == female' there are 25.8% of your cases that are class 0 and 74.2% that are 1, as, see the above decision rule, you decide to put them all in class 1 you make an error for 25.8% of all females.

EDIT; because of the questions in the comments I added:

The underlying algorithm in rpart that divides the tree into branches. The algorithm tries to make each leaf node 'as pure as possible', i.e. with as much possible cases of the same observed class in the leaf node. Therefore these algorithms use an 'impurity measure', there exists more than one impurity measure, for details see section 9.2.3 of this pdf