Almost all the examples I have found stated how the decision tree's split is based on how much purity/information can be gained (ie: via entropy and information gain) for internal node. But is the same procedure done to determine the root node as well, or is other procedure employed to determine the root node?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ As ID3 and C4.5 does greedy search, all the 10000 features are tested at the root node. $\endgroup$– JalendarCommented Nov 15, 2018 at 12:06
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
It's the same procedure. The information gain rule applies regardless of the node level. There is nothing special about the root node.
answered Jul 7, 2016 at 1:15
-
$\begingroup$ Just to dive a bit deeper. Say, there are 10000 features (with Yes or No value), does the root node test the information gain on each of these 10000 features to determine which is the best root node? Or does it randomly sample a subset for efficiency at the expense of not finding the best root node? $\endgroup$– KubiK888Commented Jul 7, 2016 at 4:00
-
$\begingroup$ That depends on the algorithm you use to build the tree. Most tree building algorithms will search through all of the features. The only algorithm/model that comes to mind that does not do this is called a "Random Forest", which usually samples only 1/3 of the features at each node. $\endgroup$ Commented Jul 7, 2016 at 12:01
-
$\begingroup$ @Ryan Zotti According to Ryan's answer, root node is determined by information gain rule not random? I'm confused that The only algorithm/model that comes to mind that <does not do> this is called a "Random Forest". In other words, do you understand does not do this called a Random Forest? $\endgroup$– DSDSCommented Dec 30, 2017 at 7:03