0
$\begingroup$

I am trying to understand how a CART tree grows, So I am growing a tree step by step, and I am finding a strange (?) behavior. Let me show this by means of an example: I will use the titanic data set and the rpart package to grow the tree. Here is the code:

#######################
###load packages
#######################
library(tidyverse)
library(rpart)
library(partykit)
#> Loading required package: grid
#> Loading required package: libcoin
#> Loading required package: mvtnorm
library(titanic)
library(ggparty)

z0<-titanic_train

###formating factors
for(i in c("Survived","Pclass","Sex","Embarked")){
  z0[,i]<- factor(z0[,i])
}
z0[,"Survived"]<-factor(z0[,"Survived"],labels = c("No","Yes"))

#First split
fitTT1<-rpart(Survived ~ Pclass+Age+Sex+Fare, data = z0,
              control=rpart.control(maxdepth=1))
fitTT1 %>% as.party() %>%autoplot


#2nd split
fitTT2<-rpart(Survived ~ Pclass+Age+Sex+Fare, data = z0,
              control=rpart.control(maxdepth=2))
fitTT2 %>% as.party() %>%autoplot


#3rd split
fitTT3<-rpart(Survived ~ Pclass+Age+Sex+Fare, data = z0,
              control=rpart.control(maxdepth=3))
fitTT3 %>% as.party() %>%autoplot

Created on 2023-12-07 with reprex v2.0.2

As you can see, the first split is based on Sex, and in the second split, only the node associated with males is split but not the node of females. In the third split, now the female node is split in three nodes.

My question is why in the second round of splits the female node is not split in the same way as the male node since variable Pclass can make a split in this node as well. How is this parameter maxdepth working? Why is the split of females not considered as a split of depth 2? Is there a way to obtain a tree with four leafs: males-Age>6.5, male-Age<6.5, female-Pclass3, female-Pcalss1,2?

$\endgroup$

2 Answers 2

2
$\begingroup$

You wrote

As you can see, the first split is based on Sex, and in the second split, only the node associated with males is split but not the node of females. In the third split, now the female node is split in three nodes.

No, the third split splits females into 2 nodes based on PClass. The fourth split splits females in Pclass 3 into 2 nodes based on fare. You set a maximum depth not a maximum number of splits.

why in the second round of splits the female node is not split in the same way as the male node

Because splitting females based on age did not work as well as splitting them based on class. This ability to split different nodes on different variables is not bug, it's a feature. I'd say it's one of the big features of trees and is hard to replicate in other methods.

How is this parameter maxdepth working?

It is working by limiting the depth of the tree. It is not going to try to make further splits on females of Pclass 3 and either low fare or high fare. Maybe one of those splits would be good, but you told it not to look. For the other terminal nodes, it looked for good splits, but didn't find them.

Why is the split of females not considered as a split of depth 2?

I'm not sure exactly what you mean here, but it starts at depth = 0. Is that what's confusing you?

Is there a way to obtain a tree with four leafs: males-Age>6.5, male-Age<6.5, female-Pclass3, female-Pcalss1,2?

You could try maxdepth = 2. I'm pretty sure that will do it.

$\endgroup$
2
  • $\begingroup$ depth 1 produced leaves based on sex. depth 2 produced 3 leaves: males-Age>6.5, male-Age<6.5, females. depth 3 produced 5 leaves as the graph shows. I have a hard time understanding why females didn't split at depth 2, but at depth 3 and not with one but two splits. most of the time you read in the books or papers that splits are attempted in each node, so I was expecting sequential splits as depth of the tree increases. your last suggestion is already in the code by the way. $\endgroup$ Dec 7, 2023 at 19:06
  • $\begingroup$ Oh, sorry about my mistake. But trees algoritms often grow the tree to a much higher depth and then prune back, so the best tree at depth 3 may include an earlier split that depth 2 did not find. $\endgroup$
    – Peter Flom
    Dec 7, 2023 at 20:08
0
$\begingroup$

This post is of great help to answer this.

The reason why the female node didn't split at maxdepth 2 was because rpart (CART algorithm implemented in R) 1) prevents a split to be made if complexity is not above some threshold and 2) performs a pruning before the tree is grown by means of cross-validation. In the aforementioned post, it is stated how to set appropriate values to prevent this pruning and let the tree grow regardless of complexity gains.

Here I reproduce the code of the post, just for easy of presentation.

fitTT2 <- rpart(
  Survived ~ Pclass + Age + Sex + Fare, 
  data = z0,
  maxdepth = 2,
  method = "class", 
  parms = list(split = "information"),
  xval = 0,
  cp = -1
)

fitTT2 |> 
  as.party() |> 
  autoplot()

tree with required split

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.