In decision tree classification, we use the attribute that splits records, like entropy, as split nodes.

Does it need to consider the correlation between attributes?

  • $\begingroup$ Usually, the algorithms used will automatically take into accoiunt any feature correlations. $\endgroup$ Commented Feb 13, 2015 at 18:53
  • $\begingroup$ We know that Bayes classifier must use the independent attributes as input. Does Decision tree have the restrictions? $\endgroup$
    – WeiYuan
    Commented Feb 14, 2015 at 1:12
  • $\begingroup$ In addition, if we use dependent attributes in decision tree classification, the algorithm will automatically handle it? Can the entropy consider the correlation between attributes? $\endgroup$
    – WeiYuan
    Commented Feb 14, 2015 at 1:14

1 Answer 1


Already asked here Decision trees recursively partitions the input feature/attribute space to reduce the entropy/impurity score. The output on an unseen sample is obtained by finding the leaf node/ decision region containing the sample and calculating the majority class/label withing this leaf node. Here and in the original CART trees as well as the implementation in scikit-learn do the take into account the correlation between the attribute variables.

While we do know that this is taken care of by random subspace selection in random forests, which are an ensemble of decision trees. This consists of performing randomly subsampling the attributes without replacement.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.