Any significance of area under curves in lasso plot?

Question

Following plot is obtained on performing LASSO using glmnet package:

Is there any significance of area under the curves (using 0 as baseline) in reporting significance of the variables? Can we say that importance of different variables in predicting the dependent variable is reflected by the area under its curve? So here the purple and black variables are probably equally important. They are followed by (in descending order) deep blue, green, light blue and red variables.

Answer 1

A couple things that immediately occur to me about this.

I think spdrnl's right, due to the standardization, the effect sizes should be comparable. It looks like it may be the case that the plot is on the scale of the original variables though, I'd check which is true and work with a plot of the coefficients of the standardized predictors.

First observation. I think you'll want to be careful with your region of integration. Suppose the most predictive model is associated with a $\log(\lambda)$ somewhere in the middle of the plot. Then the models corresponding to the left hand side of the plot are overfit, and just capturing noise in the data. You probably don't want to report on this area. So, in terms of lambda, I would recommend integrating:

$$ \int_0^{\lambda_{opt}} | \beta_i(t) | $$

Second observation. You are going to lose some subtlety with non-monotonic coefficient paths. I'm thinking of your lasso example from yesterday

Here the area method would report some definite significance for cyl. What's really true is that cyl is important for small models, then the effect drops out for large models. The area approach does not capture this. You may want to complement your area measurements with comments or pictures focusing on these interesting cases.

Finally, you'll have to choose what to measure on your x-axis. The choices are $\lambda$, $\log(\lambda)$ and $\sum_i | \beta_i |$. I would lean towards the latter, as that is measuring how much of the total allocated coefficient budget goes to each predictor. The others are only interpretable though Lagrange multipliers, making it hard to really be sure what is being measured.