0
$\begingroup$

I wonder if anyone can comment on if the following modelling strategy is valid please? I have a 200 patient survival data set (actually 2 data sets: 40 events and 160 events) and 100,000 ish candidate predictors.

I want to build a prediction model so I plan to use LASSO and elastic net (with glmnet). I was intending to split into 2/3 train, find the best shrinkage lambda (via CV in the training data) and predict on the remaining 1/3 (by c-statistic).

I was planning on say 50 test training splits to see if the same predictors were selected repeatedly and the variability in best lambda and estimated c-statistics in the test dataset.

I was then going to obtain a final model using all the data and some shrinkage parameter (maybe the mean of the best lambdas from the 50 splits - I'd hope they were all similar) I'd then claim the c-statistic of that final model is within the range of the c-statistics found from the 50 train/test splits (maybe around the mean). Are there are flaws in this scheme or room for improvement ??

many thanks in advance

$\endgroup$
0
$\begingroup$

just thought when I said "I was then going to obtain a final model using all the data and some shrinkage parameter" - that it would obviously be better to select that shrinkage parameter from cross validating the data set (just like I did with the 50 traininng data sets). a side issue is cross-validation in glmet for Cox is based on partial likelihood whereas I'd be interested in the c-statistic so I;d have no c-statistic for that final model. I suppose I can see that, adjusted for overfitting, it'l be similar to the (mean of) those from the 50 test data sets but I'd have thought being based on more data (the entire data set) it might be better (I'll never know as I'd have no "new data" to apply it to)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.