1
$\begingroup$

  • Sorry, but this is not a general question, so i am going to be as specific as i can. I have searched a lot, however i cant consider the following case in regression trees:

  • The following picture shows one node, which has 2 Samples. The black rectangles are the split nodes and the green circles are the leaf nodes

Picture of a two observation node in a tree being split into two one observation nodes

  • In each node, the split is made by using the SSE(Residual Sum Of Square Error)

  • I have problem in calculating the split, when the num of samples in a node is 2. This happens because the SSE calculates the mean values in each subtree of this node. So, the SSE will be always zero

(leftSubtreeError-meanLeft)^2 + (rightSubtreeError-meanRight)^2 = (leftSubtreeError-leftSubtreeError)^2 + (rightSubtreeError-rightSubtreeError)^2 = 0^2 + 0^2 = 0

  • Also each leaf node will always have 1 sample

  • The split is done according to 3 variables:

    a. px1(height,width)

    b. px2(height,width)

    c. thres, where 1 <= thres <= 50

  • So, if the number of samples in a node is 2, does it make sense using the above split variables? no

  • Maybe i am missing some part of algorithm. Is there any exception in the algorithm?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

In the situation you have outlined a basic implementation of a decision tree will result in two nodes with one training observation each, at least as long as there is some feature that discriminates between those two observations.

It is possible, and relatively common, to add safeguards against this kind of situation, this amounts to a form of regularization. For example, you could automatically reject any proposed split on a node withe less than $N$ observations, or disallow the creating of any leaf nodes with less than $N$ observations.

But if you don't do that, then you'll get two nodes, each containing one observation.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Yes this solves the problem. The simplest way is to choose N=2 i guess. I could probably experiment to find the optimal N value $\endgroup$
    – trakis95
    Commented Jul 16, 2018 at 11:05
  • $\begingroup$ Basically the reason why this occured was that the num of split variables where a lot more than the num of samples in the every last split node. So, some of the variables were never examined or were useless $\endgroup$
    – trakis95
    Commented Jul 26, 2018 at 18:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.