4
$\begingroup$

In the elastic net, and specifically in glment package in R - how would I obtain the number of degrees of freedom? Note that in the Glmnet Vignette it says that the df you get from the ouput after an Elastic Net model is the number of effective degrees of freedom for the LASSO! I'm using elastic net with alpha = 0.5, so the question is what do I use for the degrees of freedom? (I need that to compute AIC manually)

$\endgroup$
3
$\begingroup$

The answer to your question is here:

enter image description here

This is from a talk by Hui Zou. You can find the full talk here.

This should be relatively easy to implement in R. If you need some guidance, you can check the lassovar package by A. Kock and L. Callot; see the command .ridge.df in lassovar-ada.R.

Related reference:

Zou, H., Hastie, T., & Tibshirani, R. (2007). On the "degrees of freedom" of the lasso. Ann. Statist., 35(5), 2173–2192. https://doi.org/10.1214/009053607000000127 -- This is the formal proof that df of the lasso = number of non-zero coefficients of the lasso.

$\endgroup$
3
  • $\begingroup$ If this is the correct formula, I was wondering why glmnet is just returning the nr of nonzero coefficients as $df though. Would anyone know? $\endgroup$ – Tom Wenseleers Apr 14 '18 at 7:35
  • $\begingroup$ I suppose returning the effective df would be computationally costly as it requires one matrix inversion per lambda. Also, the focus of glmnet is on cross-validation rather than tuning parameter selection via information criteria (for which df are relevant). $\endgroup$ – aahr1 Apr 15 '18 at 10:53
  • $\begingroup$ Is this the true active set or the estimated active set? Using the true active set would make the estimator infeasible, so I suppose we have the estimated one here. $\endgroup$ – Richard Hardy May 3 '19 at 14:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.