I am running a ridge regression using GLMNET (alpha = 0) and would like to interpret the coefficients returned. I know there isn't really a significance test for this, but can I at least rank the variable importance? I am interested in explanatory, not predictive power, which is why this is important to me.
Here are some of my thoughts on how to do this:
- Standardize the data before running it through GLMNET. Pass
standardize=Fto GLMNET and just sort the coefficients by magnitude. I'm not sure that this is correct, but someone suggested it for LASSO elsewhere.
- If I run the regression using caret, then it gives me a
varImpfunction. I don't know how it calculates this, but the results seem nice. How does this work, and if it is correct, can I implement it for standard GLMNEt without caret?
- Someone recommended I somehow compute confidence intervals for each coefficent and see how far they are from 0. Any variable that included 0 in this interval is unimportant.
One problem here is that options 1 and 2 are giving me different results, so I am not sure who to trust.
Edit: I see from this answer that in option 2, caret's
varImp function is actually just the magnitude of the coefficients (option 1).