In the elastic net, and specifically in glment package in R - how would I obtain the number of degrees of freedom? Note that in the Glmnet Vignette it says that the df you get from the ouput after an Elastic Net model is the number of effective degrees of freedom for the LASSO! I'm using elastic net with alpha = 0.5, so the question is what do I use for the degrees of freedom? (I need that to compute AIC manually)
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
The answer to your question is here:
This is from a talk by Hui Zou. You can find the full talk here.
This should be relatively easy to implement in R. If you need some guidance, you can check the lassovar package by A. Kock and L. Callot; see the command .ridge.df in lassovar-ada.R.
Related reference:
Zou, H., Hastie, T., & Tibshirani, R. (2007). On the "degrees of freedom" of the lasso. Ann. Statist., 35(5), 2173–2192. https://doi.org/10.1214/009053607000000127 -- This is the formal proof that df of the lasso = number of non-zero coefficients of the lasso.
-
1$\begingroup$ If this is the correct formula, I was wondering why glmnet is just returning the nr of nonzero coefficients as $df though. Would anyone know? $\endgroup$ Commented Apr 14, 2018 at 7:35
-
$\begingroup$ I suppose returning the effective df would be computationally costly as it requires one matrix inversion per lambda. Also, the focus of glmnet is on cross-validation rather than tuning parameter selection via information criteria (for which df are relevant). $\endgroup$– aahr1Commented Apr 15, 2018 at 10:53
-
$\begingroup$ Is this the true active set or the estimated active set? Using the true active set would make the estimator infeasible, so I suppose we have the estimated one here. $\endgroup$ Commented May 3, 2019 at 14:40