3
$\begingroup$

I am trying to understand rpart all the details in rpart package. I am aware of the complexity parameter cp, which prevents a split if the improvement is less than cp

If I set minbucket = 1, minsplit = 1 and cp = -Inf (or cp = 0) the tree should be allowed to grow to perfectly fit the data; provided that all the values of the predictor, or combination of predictors, are different. But it does not.

There must be another parameter which is preventing some splits to be made since, as you can see in this image, data is not fitted perfectly. One can clearly see there are some leaves which have more than one element, as we see them layered in the lower part.

enter image description here

This is a MVE illustrating this issue, and generating the image above:

set.seed(1)
sample_size <- 1000
y <- rgamma(sample_size, shape = 2, rate = 0.75)
x <- rgamma(sample_size, shape = 0.5, rate = 2)

library(rpart)
md <- rpart(formula = y ~ x, data = data.frame(y,x), method = "anova", cp = -Inf, minbucket = 1, minsplit = 1)

# All elements in x are unique:
length(unique(x)) == sample_size

#Number of leaves with more than one element:
sum(md$frame[md$frame$var == "<leaf>", "n"] > 1)

#Scatterplot
plot(y, predict(md, newdata = data.frame(x)), xlab = "Observed", ylab = "Predicted", 
     col = scales::alpha("black", 0.2), pch = 16)

Note: For debugging purposes note that using sample_size = 185 already produces a leaf with 2 values.

$\endgroup$
2
  • $\begingroup$ Changing the sample size from 1000 to 185 already yields one leaf with more than one observation. A smaller MWE might make analysis easier. $\endgroup$ Commented Nov 3, 2020 at 12:20
  • $\begingroup$ @StephanKolassa Great suggestion, I have added a note. I have left the original value since the scatterplot is more clear. $\endgroup$
    – D1X
    Commented Nov 3, 2020 at 12:30

1 Answer 1

1
$\begingroup$

I apologize for posting an erroneous answer in a former attempt to help (now deleted). This post explains how to calculate the depth of a tree: https://stackoverflow.com/a/40900725/6503141

So if you run the code in the question and after that the following

nodes <- as.numeric(rownames(md$frame))
max(rpart:::tree.depth(nodes))

You will find that you have reached the maxdepth = 30 limit described in help(rpart.control) as

Set the maximum depth of any node of the final tree, with the root node counted as depth 0. Values greater than 30 rpart will give nonsense results on 32-bit machines.

On my 64-bit Windows machine (R version 4.0.2, rpart version 4.1-15) setting maxdepth to anything larger then 30 leads to an error message. With set.seed(1); sample_size <- 185 you also reach maxdepth == 30 but not with set.seed(1); sample_size <- 184. This makes maxdepth the a likely candidate for the parameter you search. Unfortunately there seems to be no obvious way to increase it.

$\endgroup$
1
  • $\begingroup$ Seems like we have a smoking gun. I will do some tests to confirm this. Thanks a lot. $\endgroup$
    – D1X
    Commented Nov 5, 2020 at 9:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.