Understanding Partial Likelihood Deviance vs lambda relationship in LASSO

Question

I'm analyzing gene expression data using regularized linear regression models (lasso-elastic net-ridge) and would like to interpret the relationship between genes (predictor variable, scaled) and various clinical parameters (response variables). These response variables include survival outcome and clinical classifiers (tumor type etc). I checked out the glmnet vignette and Tibshirani/Hastie's Introduction to Statistical Learning books and lectures. My question has to do with interpreting the relationship between lambda and the deviance.

In the glmnet vignette, I saw cv.glmnet() survival cross-validation output produces the following plot:

My data shows an inverse trend though:

I'm trying to understand why this is the case. I would have expected that, with the increasing lambda (ie. increased penalization of coefficients), the deviance should increase (ie. less biased fit), similar to what is seen in the glmnet vignette. Why is my data showing this trend? Also, what do the numerical values of lambda and Partial Likelihood Deviance tell me about the linear regression model?

Answer 1

It's two very different datasets, if you run the example in the vignette:

data(CoxExample)
dim(y)
[1] 1000    2
dim(x)
[1] 1000   30

There's only 30 genes in the example and looking at the numbers on top of your plot, you have > 100, so the range of lambda tested goes lower.

The range of lambda goes all the way to 10^-8 whereas in the first plot its 10^-5. The range of the likelihood deviance is much larger too in the first plot.

You can check out this publication where the authors apply glmnet cox on a dataset similar to yours. They performed CV and have a plot similar to yours, and then zoomed in on the lower deviance part.

I guess it makes sense for this kind of data because a lot of the genes are correlated and if you go with no penalization, fitting all of them gives a high deviance. Another thing to take note of in your plot and theirs is the number on top which reflects the non-zero coefficients. It's actually around the same (~30) which suggest having about 30 genes provides a reasonable prediction.