1
$\begingroup$

I read that some techniques of feature selection, like backward feature elemination use p-values of different features. But how is this p-value claculated ? I searched this question a lot and just got the information of the null hypothesis (the feature is not significant) and alternate hypothesis (the feature is significant), which was obvious.

So my question is how are these p-values calculated ? Is this done using the coefficients which we get after training the model ? If yes , then which test is used to get p-value?

$\endgroup$
1
$\begingroup$

How feature significance is determined is dependent upon the model of interest. For a standard LM, it will be chi-squared or a t-test, for a GLM, usually a z-test. For comparing whether one model is better than another, which is another common thing forwards/backwards selection tools do, it will usually be a chi-squared test, an F-test, a LRT, or just comparison of AICs/BICs of the resulting model. Note again that "which" will be dependent on the model. If you are using someone else's code (such as stats package from R) it will tell you in the summary what was done, more often than not, and should be in the resulting data object you obtain after fitting. If it is not, I would check through the documentation for the function you are calling. Finally, re: "is this done using the coefficients which we get after training the model?" the answer is something between "usually" and "yes"; most stuff will compute p-values for features of interest (or, for outright model comparison, as mentioned) after fitting is done.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.