0
$\begingroup$

Setup

Consider two trees grown with rpart() from the rpart R package, in which the only thing that changes is the minbucket argument.

Using the same data, I do not understand why some split is not performed on a given node despite a potential split seems possible. By lowering the minbucket argument, the resulting split shows that the children actually both have more than the minimal number of observation in them.

library(rpart)
data(iris)
# First Tree: minbucket = 20
tree_1 <- rpart(
  Species ~ ., method='class', data = iris,
  control = rpart.control(xval = 0, minbucket = 20, cp = -1)
)

# Second Tree: minbucket = 15
tree_2 <- rpart(
  Species ~ ., method='class', data = iris,
  control = rpart.control(xval = 0, minbucket = 15, cp = -1)
)

The resulting trees, for the first one:

> tree_1
n= 150 

node), split, n, loss, yval, (yprob)
      * denotes terminal node

1) root 150 100 setosa (0.33333333 0.33333333 0.33333333)  
  2) Petal.Length< 2.45 50   0 setosa (1.00000000 0.00000000 0.00000000) *
  3) Petal.Length>=2.45 100  50 versicolor (0.00000000 0.50000000 0.50000000)  
    6) Petal.Width< 1.75 54   5 versicolor (0.00000000 0.90740741 0.09259259) *
    7) Petal.Width>=1.75 46   1 virginica (0.00000000 0.02173913 0.97826087) *

The second one:

> tree_2
n= 150 

node), split, n, loss, yval, (yprob)
      * denotes terminal node

 1) root 150 100 setosa (0.33333333 0.33333333 0.33333333)  
   2) Petal.Length< 2.45 50   0 setosa (1.00000000 0.00000000 0.00000000) *
   3) Petal.Length>=2.45 100  50 versicolor (0.00000000 0.50000000 0.50000000)  
     6) Petal.Width< 1.75 54   5 versicolor (0.00000000 0.90740741 0.09259259)  
      12) Petal.Length< 4.45 29   0 versicolor (0.00000000 1.00000000 0.00000000) *
      13) Petal.Length>=4.45 25   5 versicolor (0.00000000 0.80000000 0.20000000) *
     7) Petal.Width>=1.75 46   1 virginica (0.00000000 0.02173913 0.97826087)  
      14) Sepal.Length< 6.35 16   1 virginica (0.00000000 0.06250000 0.93750000) *
      15) Sepal.Length>=6.35 30   0 virginica (0.00000000 0.00000000 1.00000000) *

Question

With the first tree, where minbucket = 20, node 6 is terminal. It contains 54 observations. With the second tree, where minbucket = 15, node 6 is no longer terminal and is split into the children 12 and 13, which contains 29 and 25 observation, respectively.

Why was the split not made for node 6 in the first tree, even if the children would both contain more than 20 observation each?

Additional Information

I have looked at the Gini impurity in node 6 as well as the weighted sum of the Gini impurity in nodes 12 and 13. There is a gain of 0.1183673.

library(dplyr)
node_6 <- iris |> filter(Petal.Length >= 2.45, Petal.Width < 1.75)
node_12 <- node_6 |> filter(Petal.Length < 4.45)
node_13 <- node_6 |> filter(Petal.Length >= 4.45)

#' Calculate Gini impurity for a given set of labels
#'
#' @param labels vector of labels
calculate_gini_impurity <- function(labels) {
  label_counts <- table(labels)
  proportions <- label_counts / sum(label_counts)
  gini_impurity <- 1 - sum(proportions^2)
  gini_impurity
}

# Gini Impurity in Node 6
gini_6 <- calculate_gini_impurity(node_6$Species)
# Gini Impurity in Nodes 12 and 13
gini_12 <- calculate_gini_impurity(node_12$Species)
gini_13 <- calculate_gini_impurity(node_13$Species)
weights <- c(nrow(node_12), nrow(node_13)) / nrow(node_6)
gini_tot <- (gini_tot_children <- weights %*% c(gini_12, gini_13))

The values:

> c(Gini = gini_6, Gini_Tot = gini_tot)
     Gini  Gini_Tot 
0.1680384 0.1481481 
$\endgroup$

1 Answer 1

1
$\begingroup$

The documentation says

minsplit the minimum number of observations that must exist in a node in order for a split to be attempted. minbucket the minimum number of observations in any terminal node. If only one of minbucket or minsplit is specified, the code either sets minsplit to minbucket*3 or minbucket to minsplit/3, as appropriate

If you set both minbucket and minsplit you can get what you were looking for, eg,

> tree_3<-rpart(
+   Species ~ ., method='class', data = iris,
+   control = rpart.control(xval = 0, minbucket = 20, minsplit=50,cp = -1)
+ )
> tree_3
n= 150 

node), split, n, loss, yval, (yprob)
      * denotes terminal node

 1) root 150 100 setosa (0.33333333 0.33333333 0.33333333)  
   2) Petal.Length< 2.45 50   0 setosa (1.00000000 0.00000000 0.00000000) *
   3) Petal.Length>=2.45 100  50 versicolor (0.00000000 0.50000000 0.50000000)  
     6) Petal.Width< 1.75 54   5 versicolor (0.00000000 0.90740741 0.09259259)  
      12) Petal.Length< 4.45 29   0 versicolor (0.00000000 1.00000000 0.00000000) *
      13) Petal.Length>=4.45 25   5 versicolor (0.00000000 0.80000000 0.20000000) *
     7) Petal.Width>=1.75 46   1 virginica (0.00000000 0.02173913 0.97826087) *

splits node 6 but not node 7

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.