Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I'm a bit confused how decision tree's select the variables to split. I know they splitt the data set through variable to get a more pure data set. But can it happend that some explenatory variables or some classes of a categorical variable that are not used in the construction of the tree because they dont improve the impurity of the data set are dropped out?
So do decision tree's perform a variable selection and can drop some variable that are not usefull for the precidion?
Decision trees have the ability to not to choose some set of features if these do not contribute to the impurity or entropy (based on the implementation) decrease. So, this can be regarded as some kind of basic feature selection/dropping scheme, while since the split algorithms are typically greedy, some moderately useful features may be labeled as not so useful, because much of the work has been done by other features up to that extent.
