2
$\begingroup$

I know decision trees commonly reuse continuous features down a particular branch of the tree and splitting them in different locations. I am wondering what the downsides are to disallowing this?

For context, I am attempting to use a binary classification decision tree to create a white-box model of a feature hierarchy. Top node would then be the feature that gives you the most information gain about the binary classification. This will also establish a context, basically which feature will give us the next highest information gain conditioned on our current path.

In this hierarchy, reusing features seems like it would disrupt the actual hierarchy which is why I want to disallow reuse of features.

$\endgroup$

1 Answer 1

2
$\begingroup$

Any time you have the true/generative model having some sort of curvature in your predictor space (e.g. $x_i^2$ or interactions of the form $x_i\times x_j$) you cannot capture/approximate this behavior with a single split in a single decision tree. Allowing for multiple splits on the same feature allows the decision tree to approximate the curvatures inherent in the feature space. In machine learning learning the basis of the feature space is often referred to as the kernel method.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.