Skip to main content
added 153 characters in body
Source Link
D1X
  • 773
  • 1
  • 7
  • 23

I am trying to understand rpart all the details in rpart package. I am aware of the complexity parameter cp, which prevents a split if the improvement is less than cp

If I set minbucket = 1, minsplit = 1 and cp = -Inf (or cp = 0) the tree should be allowed to grow to perfectly fit the data; provided that all the values of the predictor, or combination of predictors, are different. But it does not.

There must be another parameter which is preventing some splits to be made since, as you can see in this image, data is not fitted perfectly. One can clearly see there are some leaves which have more than one element, as we see them layered in the lower part.

enter image description here

This is a MVE illustrating this issue, and generating the image above:

set.seed(1)
sample_size <- 1000
y <- rgamma(1000sample_size, shape = 2, rate = 0.75)
x <- rgamma(1000sample_size, shape = 0.5, rate = 2)

library(rpart)
md <- rpart(formula = y ~ x, data = data.frame(y,x), method = "anova", cp = -Inf, minbucket = 1, minsplit = 1)

# All elements in x are unique:
length(unique(x)) == sample_size

#Number of leaves with more than one element:
sum(md$frame[md$frame$var == "<leaf>", "n"] > 1)

#Scatterplot
plot(y, predict(md, newdata = data.frame(x)), xlab = "Observed", ylab = "Predicted", 
     col = scales::alpha("black", 0.2), pch = 16)

Note: For debugging purposes note that using sample_size = 185 already produces a leaf with 2 values.

I am trying to understand rpart all the details in rpart package. I am aware of the complexity parameter cp, which prevents a split if the improvement is less than cp

If I set minbucket = 1, minsplit = 1 and cp = -Inf (or cp = 0) the tree should be allowed to grow to perfectly fit the data; provided that all the values of the predictor, or combination of predictors, are different. But it does not.

There must be another parameter which is preventing some splits to be made since, as you can see in this image, data is not fitted perfectly. One can clearly see there are some leaves which have more than one element, as we see them layered in the lower part.

enter image description here

This is a MVE illustrating this issue, and generating the image above:

set.seed(1)
y <- rgamma(1000, shape = 2, rate = 0.75)
x <- rgamma(1000, shape = 0.5, rate = 2)

library(rpart)
md <- rpart(formula = y ~ x, data = data.frame(y,x), method = "anova", cp = -Inf, minbucket = 1, minsplit = 1)

# All elements in x are unique:
length(unique(x))

#Number of leaves with more than one element:
sum(md$frame[md$frame$var == "<leaf>", "n"] > 1)

#Scatterplot
plot(y, predict(md, newdata = data.frame(x)), xlab = "Observed", ylab = "Predicted", 
     col = scales::alpha("black", 0.2), pch = 16)

I am trying to understand rpart all the details in rpart package. I am aware of the complexity parameter cp, which prevents a split if the improvement is less than cp

If I set minbucket = 1, minsplit = 1 and cp = -Inf (or cp = 0) the tree should be allowed to grow to perfectly fit the data; provided that all the values of the predictor, or combination of predictors, are different. But it does not.

There must be another parameter which is preventing some splits to be made since, as you can see in this image, data is not fitted perfectly. One can clearly see there are some leaves which have more than one element, as we see them layered in the lower part.

enter image description here

This is a MVE illustrating this issue, and generating the image above:

set.seed(1)
sample_size <- 1000
y <- rgamma(sample_size, shape = 2, rate = 0.75)
x <- rgamma(sample_size, shape = 0.5, rate = 2)

library(rpart)
md <- rpart(formula = y ~ x, data = data.frame(y,x), method = "anova", cp = -Inf, minbucket = 1, minsplit = 1)

# All elements in x are unique:
length(unique(x)) == sample_size

#Number of leaves with more than one element:
sum(md$frame[md$frame$var == "<leaf>", "n"] > 1)

#Scatterplot
plot(y, predict(md, newdata = data.frame(x)), xlab = "Observed", ylab = "Predicted", 
     col = scales::alpha("black", 0.2), pch = 16)

Note: For debugging purposes note that using sample_size = 185 already produces a leaf with 2 values.

Source Link
D1X
  • 773
  • 1
  • 7
  • 23

Why does rpart not produce a perfect prediction when forced to?

I am trying to understand rpart all the details in rpart package. I am aware of the complexity parameter cp, which prevents a split if the improvement is less than cp

If I set minbucket = 1, minsplit = 1 and cp = -Inf (or cp = 0) the tree should be allowed to grow to perfectly fit the data; provided that all the values of the predictor, or combination of predictors, are different. But it does not.

There must be another parameter which is preventing some splits to be made since, as you can see in this image, data is not fitted perfectly. One can clearly see there are some leaves which have more than one element, as we see them layered in the lower part.

enter image description here

This is a MVE illustrating this issue, and generating the image above:

set.seed(1)
y <- rgamma(1000, shape = 2, rate = 0.75)
x <- rgamma(1000, shape = 0.5, rate = 2)

library(rpart)
md <- rpart(formula = y ~ x, data = data.frame(y,x), method = "anova", cp = -Inf, minbucket = 1, minsplit = 1)

# All elements in x are unique:
length(unique(x))

#Number of leaves with more than one element:
sum(md$frame[md$frame$var == "<leaf>", "n"] > 1)

#Scatterplot
plot(y, predict(md, newdata = data.frame(x)), xlab = "Observed", ylab = "Predicted", 
     col = scales::alpha("black", 0.2), pch = 16)