1) How do we decide if we split into 2 subnodes, or more? Or is it always 2?

2) How do we decide what threshold is the cutoff? Specifically, you have a continuous variable, do you do a binary search, or how is the cutoff decided?


Well, it depends on the implementation you are using. I assume we are talking about the original CART paper [1]

1) then there is always a single split resulting in two children.

2) The value used for splitting is determined by testing every value for every variable, that the one which minimizes the sum of squares error (SSE) best is chosen:

$SSE=\sum_{i\in S_1}({y_i- \bar{y}_1})^2+\sum_{i\in S_2}({y_i- \bar{y}_2})^2$

In the equation above $y_i$ is your predictors value and $\bar{y}_1$ and $\bar{y}_2$ the mean value of the left and right hand side of the possible split.

When feeding the following data matrix to the CART routine, every value for every variable/feature/column (here A,B and C) would be tested using SSE.

    A     B     C     y    
    0.05  0.31  0.51  0.97 
    0.32  0.41  0.88  0.89 
    0.76  0.61  0.48  0.11 
    0.81  0.94  0.85  0.19

The one minimizing SSE best, would be chosen for split. CART would test all possible splits using all values for variable A (0.05, 0.32, 0.76 and 0.81) and then using variable B, then C.

[1] Breiman, Leo, et al. Classification and regression trees. CRC press, 1984.

  • $\begingroup$ How can it try every value if the feature we are splitting on is continuous? $\endgroup$ – The Baron Jun 23 '16 at 18:05
  • $\begingroup$ Just edited my answer. Hope it's a bit more clear now. $\endgroup$ – mariodeng Jun 23 '16 at 18:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.