1
$\begingroup$

I'm a bit confused how decision tree's select the variables to split. I know they splitt the data set through variable to get a more pure data set. But can it happend that some explenatory variables or some classes of a categorical variable that are not used in the construction of the tree because they dont improve the impurity of the data set are dropped out?

So do decision tree's perform a variable selection and can drop some variable that are not usefull for the precidion?

Improve this question
$\endgroup$
0
$\begingroup$

Decision trees have the ability to not to choose some set of features if these do not contribute to the impurity or entropy (based on the implementation) decrease. So, this can be regarded as some kind of basic feature selection/dropping scheme, while since the split algorithms are typically greedy, some moderately useful features may be labeled as not so useful, because much of the work has been done by other features up to that extent.

Improve this answer
$\endgroup$
2
  • $\begingroup$ thx for the answer, have you maybe a paper where is written what you said, especially the party of the ,,droping sheme '' that i can cite in my thesis? $\endgroup$ – MasterStudent1992 Apr 6 '19 at 23:48
  • 1
    $\begingroup$ Maybe this?: fizyka.umk.pl/publications/kmk/05-Fsel-DT.pdf (e.g. Intro 2nd paragraph) $\endgroup$ – gunes Apr 7 '19 at 9:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.