0
$\begingroup$

Im new to decision trees and am trying some decision tree modelling now with titanic data.

I have the following dataset.

 train <- read.csv(url("http://s3.amazonaws.com/assets.datacamp.com/course/Kaggle/train.csv"))

Now when I create a decision tree doing this:

my_tree_two <- rpart(Survived ~ Sex + Age, data=train, method="class")
my_tree_two

I get the following output.

n= 891 node), split, n, loss, yval, (yprob)
  * denotes terminal node

1) root 891 342 0 (0.6161616 0.3838384)  
2) Sex=male 577 109 0 (0.8110919 0.1889081)  
4) Age>=6.5 553  93 0 (0.8318264 0.1681736) *
5) Age< 6.5 24   8 1 (0.3333333 0.6666667) *
3) Sex=female 314  81 1 (0.2579618 0.7420382) *

I understand most of it but what I do not understand is why age divided into age >= 6.5 and < 6.5? Could anybody elaborate on this?

$\endgroup$
  • $\begingroup$ The algorithm always puts splits at the midpoint of two data values. $\endgroup$ – Matthew Drury Aug 17 '15 at 17:13
  • 1
    $\begingroup$ A great tutorial of the Titanic data in R was created by Trevor Stephens. Decision tree page is here $\endgroup$ – Tchotchke Aug 17 '15 at 17:15
  • $\begingroup$ @MatthewDrury thanks for your input. So this is the top and the bottom of the intervals for Male en Female divided by two? $\endgroup$ – Marc van der Peet Aug 17 '15 at 17:38
1
$\begingroup$

Apparently the goal is to split your observations in two classes, the class is given by the binary variable 'yval'

You 'predict' yval with a decision rule. Your decision rule is:

if ( sex == "male" ) { 
   # there are 577 males in your dataset
   if ( Age >= 6.5)               # amongst the 577 males you have 553 with age >= 6.5
     yval <- 0 # so the class is '0'
   else   #Age < 6.5              # amongst the 577 males you have 24 with age < 6.5
     yval<- 1
}
else  # sex is female
  # there are 314 females in your dataset
     yval <- 1.  

Obviously, the class predicted by your decision rule can be erroneous and different from the class that was observed in your dataset.

In the leaf node 'sex == female' there are 25.8% of your cases that are class 0 and 74.2% that are 1, as, see the above decision rule, you decide to put them all in class 1 you make an error for 25.8% of all females.

EDIT; because of the questions in the comments I added:

The underlying algorithm in rpart that divides the tree into branches. The algorithm tries to make each leaf node 'as pure as possible', i.e. with as much possible cases of the same observed class in the leaf node. Therefore these algorithms use an 'impurity measure', there exists more than one impurity measure, for details see section 9.2.3 of this pdf

$\endgroup$
  • $\begingroup$ @f coppens, thanks for your answer. Could you also tell me why r divides Age into >=6.5 and smaller <6.5 $\endgroup$ – Marc van der Peet Aug 17 '15 at 19:45
  • $\begingroup$ @Marc van der Peet: it is the algorithm implemented in rpart that divides the tree into branches, not R. The algorithm tries to make each leaf node 'as pure as possible', i.e. with as much possible cases of the same observed class in the leaf node. Therefore these algorithms use an 'impurity measure', there exists more than one impurity measure. Details are in section 9.2.3 of web.stanford.edu/~hastie/local.ftp/Springer/OLD/… $\endgroup$ – user83346 Aug 17 '15 at 19:52
  • $\begingroup$ @MarcvanderPeet Is the confusion that 6.5 is not a true data value, i.e. not a valid value of Age? $\endgroup$ – Matthew Drury Aug 19 '15 at 14:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.