0
$\begingroup$

I'm using ANOVA F-Value from Scikit-learn (link) to find importance of features in a machine learning task. I have numeric feature and binary classification. I'm also filtering the features whose p-value is > 0.05, but I don't know what value for the ANOVA F-Value is good enough?

$\endgroup$
4
  • $\begingroup$ Leaving aside whether what you're doing makes sense, which F corresponds to a given p-value is a function of the two degrees-of-freedom parameters. There's no single F value that goes with $p\approx 0.05$ $\endgroup$
    – Glen_b
    Aug 26, 2017 at 2:02
  • $\begingroup$ Could you explain why this approach is not making sense? If F-value is not suitable for feature importance, why it's included in the scikit-learn? $\endgroup$
    – HHH
    Aug 28, 2017 at 13:54
  • $\begingroup$ As for why it's there, that would best be asked by the people that decided to include it, but variable selection by backward elimination based on p-value is not a great way to choose variables, and p-value doesn't really measure importance. Some of the issues with such selection approaches I assume you're avoiding via cross validation or something similar, though. $\endgroup$
    – Glen_b
    Aug 28, 2017 at 14:58
  • $\begingroup$ I'm only filtering out features whose p-value is less that 0.05 because we don;t have sufficient confidence in the the F-Value for those features. After the filtering, I'm looking at the F-Value and select those that I think are more important (based on some buseiness logic). does this make sense? $\endgroup$
    – HHH
    Aug 28, 2017 at 15:10

1 Answer 1

2
$\begingroup$

I'm also filtering the features whose p-value is > 0.05

Don't do that. $p$-values of coefficients are a bad way to select models or features because they don't directly quantify any useful measures of model quality. Besides, while obtaining $p < α$ tells you that a null hypothesis is infeasible, obtaining $p > α$ doesn't much support its feasibility, so when a test fails to reject the null hypothesis, it doesn't tell you anything.

$\endgroup$
4
  • $\begingroup$ you mean I should not put any condition on p-value? how should I choose the best features then? $\endgroup$
    – HHH
    Aug 25, 2017 at 19:11
  • $\begingroup$ @H.Z. The Wikipedia article is a good place to start. But it's also worth considering whether you need to do feature selection in the first place. $\endgroup$ Aug 25, 2017 at 19:13
  • $\begingroup$ I need to do feature selection and scikit-learn provides this ANOVA f-Value as a feature selection technique, any comment? $\endgroup$
    – HHH
    Aug 25, 2017 at 19:19
  • 1
    $\begingroup$ @H.Z. I disagree with the sklearn people that an ANOVA is a good way to do feature selection, yes. $\endgroup$ Aug 25, 2017 at 19:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.