Lasso cox regression with bootstrap

Question

I'm looking at building a nomogram for cancer prognosis based on 20 variables. This will be derived from a cox ph model. In the past I used poor methodology including dichotomization and stepwise selection. I am looking to improve the methodology substantially on an upcoming study.

I understand that using Lasso for variable selection is now built into the glmnet R package. This is what I will use initially to select variables for cox regression.

However for internal validation, I understand that bootstrapping may be a superior process to k-fold cross validation.

While I am reasonably familiar with R, this methodological sequence to build the nomogram is very foreign to me. In terms of work flow, can anyone point me in the right direction on how to build a Cox PH model with Lasso that is internally validated with bootstrapping?

Answer 1

Unless you have dozens to hundreds of potential predictors (as in gene-expression studies) it would be better to develop your model with predictors based on your knowledge of the subject matter. A useful overall strategy is presented in chapter 4 of Frank Harrell's course notes or book, with specific applications to survival analysis and how to produce nomograms presented later. The rms package is the key set of tools, which includes functions to validate and calibrate your model. If you have about 15 times as many events as you have candidate predictors, then this is the way to go.

If you have a very large number of candidate predictors then you will have to use a penalized approach. As your interest is in prediction, consider whether ridge regression (also available in glmnet) might be a better choice than LASSO. The specific predictors selected by LASSO are likely to change drastically among samples, as you can see by repeating your LASSO modeling on multiple bootstrap samples from your data and noting the differences among the sets of predictors. Ridge will include all of your predictors but with the magnitudes of their coefficients reduced to avoid overfitting. If you do need to do predictor selection with LASSO, recognize from the beginning the necessarily arbitrary choices it will make from a set of correlated predictors.

A general approach to validation is first to develop your model on your full data set. Then repeat the entire model building process on multiple bootstrap samples from your data. For LASSO that would typically involve, for each bootstrap sample, cross validation to choose a penalty value to provide a model having predictors and coefficients based on that penalty. For each model based on a bootstrap sample, evaluate its performance both on that bootstrap sample and on the original data set.

The idea is that each bootstrapped sample bears a relation to the original data set as the original data set bears to the population of interest. So the performance of a model based on a bootstrap on its own bootstrapped sample can be thought of as "training-set" performance, while that on the original data can be thought of as "test-set" performance. If there are systematic differences between training-set and test-set performances, you then correct the original model accordingly for this "optimism." Ideally, with penalized methods like ridge or LASSO there should not be much "optimism" if appropriate penalty choices were made.

This does not validate your model so much as it validates your model building process. For example, if your original data sample was unrepresentative then your model still might not generalize to the population.

There are some packages in R that might help with this process: BootValidation (which seems to extend the validation procedures of the rms package to glmnet models) and c060 (which provides "functions to perform stability selection, model validation and parameter tuning for glmnet models"). I don't have direct experience with them, however. The fall-back is to take advantage of the boot package in R to automate the bootstrapping, model building, and validation tests.