When we use top down approach to induce a decision tree, we need to use some kind of splitting criteria to choose the splitting feature and splitting value at a certain internal node.
The criteria could be:
- IG
- IGR
- Gini impurity
But the feature selection seems very computationally intensive. Because we need to enumerate all the existing features and go through all its possible splitting values. And for each of the feature/splitting value, we need to calculate the criteria value and memorize it, and select the best value.
Am I correct on this? Is there anything we can do to improve it?