Effective degrees of freedom concrete example of variable distribution with Ridge

Question

I am reading a report where they used Lasso and penalty term $\lambda$. Below is a table presented:

I have a question about the DF. The original model had 16 variables and no intercept, i.e without the penalty term it would have 16 DF. In the table we see that as lambda increase, the DF decrease -> it's penalizing and the model includes less effective variables.

So just to try to understand this;

• When $\lambda = 100$, the model has set some parameters to 0 (or decreased them), and by summing the remaining proportions of each effective variable contribution we end up with 9.939085? I.e for example it could look like $Df ; x_1=0.8, x_2 = 0.75, x_3 = 0.68, x_4 = 0.6, x_5 = 0.59, x_6 = 0.78, x_7 = 0.2, x_8 = 0, x_9 = 0, x_{10} = 0.95, x_{11} = 0.87, x_{12} = 0.1, x_{13} = .79, x_{14} = .88, x_{15} = 0.978, x_{16} = 0.971085$
• If the model had an intercept, would the orinial DF w/o penalty term applied be 17?

Answer 1

A general definition of effective degrees of freedom upon penalization appears here where you'll see the trace of a matrix product. When there is no penalization this trace (sum of diagnonals) is equal to the number of parameters in the model. When there is penalization the variance of penalized parameter estimates is smaller than the variance of unpenalized estimates, and the trace is less than the apparent number of parameters. With extreme penalization all parameters go to zero and the effect d.f. is zero.

Most definitions of effective degrees of freedom ignore intercepts.

In a certain context, Tibshirani showed that the effective d.f. with lasso can be taken as the number of nonzero coefficients, in the sense that a penalized likelihood ratio $\chi^2$ statistic for testing for the presence of any associations with $Y$ has that as its d.f. That doesn't apply to ridge regression where fractional variable contributions dominate.