# Decision Tree: Adding "important" feature doesn't necessarily improve prediction

I am using a decision tree to perform binary classification. I've found that a particular feature seems to be an important one; however, keeping it in my model doesn't yield better predictions (i.e. results in a near equivalent confusion matrix as that without the feature). Does this just mean that the feature is highly correlated with another/other feature(s) already in the model?

• How do you know that it's important if it doesn't improve the model? Looking at a confusion matrix is deceptive because it only looks at $\hat{y}>c$ rather than the model's confidence in the prediction, cf. proper vs. improper scoring rules.
– Sycorax
Mar 21, 2016 at 21:32
• I have the feature importances relative to the other features used in the model. I don't have the precise calculation, but I think it demonstrates that features nearer/at the top of the tree contribute to the prediction of a larger fraction of the input. Mar 21, 2016 at 23:34