1
$\begingroup$

Almost all the examples I have found stated how the decision tree's split is based on how much purity/information can be gained (ie: via entropy and information gain) for internal node. But is the same procedure done to determine the root node as well, or is other procedure employed to determine the root node?

$\endgroup$
1
  • $\begingroup$ As ID3 and C4.5 does greedy search, all the 10000 features are tested at the root node. $\endgroup$ – Jalendar Nov 15 '18 at 12:06
4
$\begingroup$

It's the same procedure. The information gain rule applies regardless of the node level. There is nothing special about the root node.

$\endgroup$
3
  • $\begingroup$ Just to dive a bit deeper. Say, there are 10000 features (with Yes or No value), does the root node test the information gain on each of these 10000 features to determine which is the best root node? Or does it randomly sample a subset for efficiency at the expense of not finding the best root node? $\endgroup$ – KubiK888 Jul 7 '16 at 4:00
  • $\begingroup$ That depends on the algorithm you use to build the tree. Most tree building algorithms will search through all of the features. The only algorithm/model that comes to mind that does not do this is called a "Random Forest", which usually samples only 1/3 of the features at each node. $\endgroup$ – Ryan Zotti Jul 7 '16 at 12:01
  • $\begingroup$ @Ryan Zotti According to Ryan's answer, root node is determined by information gain rule not random? I'm confused that The only algorithm/model that comes to mind that <does not do> this is called a "Random Forest". In other words, do you understand does not do this called a Random Forest? $\endgroup$ – DSDS Dec 30 '17 at 7:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.