I'm trying to use penalized Cox regression to identify genes that are strongly associated with survival for further investigation via experimental studies. I have about 160 genes and 350 patients and I'm using
glmnet package in R.
glmnet package offers two different metrics for assessing model performance: deviance and Harrell's c-statistic. When I fit LASSO penalized model using deviance as the cross validation loss (CV)function, the model identifies several genes with non-zero $\beta$ values, but checking the best model's c-statistic on the overall dataset shows no predictive ability (c-stat = .5).
Although the overall goal of the analysis is feature selection for further study and not predictive performance, the lack of any predictive ability doesn't give me much confidence about the selected genes.
When I repeat the model fitting process but use the c-statistic as the CV loss function, a different set of genes are selected and the best model's c-statistic on the overall dataset shows some predictive ability (c-stat = .62).
Which metric should be used as the loss function for Cox cross validation? It's my understanding that the c-statistic should not generally be used for model selection because it is an improper scoring rule. Also using the c-statistic as a loss function and then accepting the results based on the overall c-statistic is double dipping because the c-statistic is being optimized. With these limitations in mind, are there any scenarios where using the c-statistic as the loss function is preferable? I assume there must be some scenarios or the authors wouldn't have included it in the package.