I am new to data mining and am manually implementing decision tree classification on a dataset with all continues values. A very small sample dataset of 4 attributes (columns) would be like this:

0.012  0.64   5.6     1.12   0
0.03   0.48   13.03   0.75   1
0.02   0.19   1.3     2.92   0
0.043  0.37   1.53    0.9    1
0.9    0.44   3.01    0.12   1

Last column contains given class labels. Every value of each column can be a split threshold on that column and to realize where to split, I am using minimum Gini Index following this example:

enter image description here

Assuming that I know which value is best to split on (in a given column), now how can I realize which attribute (column) should be root, left child, right child? I have no idea on how to determine the order to place attributes in the tree. Could you please let me know about this (I would appreciate it if answers come with formulas and examples on how to derive this)

  • 2
    $\begingroup$ Decision trees are often not the best choice. Is there are reason you decided to use one here? $\endgroup$ – Frank Harrell Jun 18 '18 at 11:49

Generally, you order your attributes in a decision tree according to which one has the most predictive power.

To work out which attribute has the most predictive power, you compute the cost function for each variable on its own. There are many ways to do this, I am unable to provide formulas because you haven't specified the output of your decision tree. Essentially test each variable individually and see which one gives you the best prediction accuracy on its own, that is your most predictive attribute, and so it should be at the top of your tree.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.