Predictive Accuracy of a Survival Model using Concordance Is there a preferred approach for evaluating the predictive power of a survival regression model (e.g., Weibull Accelerated Failure Time)? The metric of choice, for now, is concordance or c-index. I'm considering three possibilities for summarizing the concordance metric in a predictive/generalizable way:


*

*Using its standard error (uses the entire dataset for evaluation)

*Performing bootstrapping (uses the entire dataset for evaluation with sampling with replacement)

*Executing, possibly repeatedly, stratified k-fold cross-validation (divides entire dataset in training and validation sets, applies sampling without replacement, but with stratification based on the state 0 (suspension) or 1 (failure))


Which is the best strategy? And why? What if the number of failures is relatively much smaller than that of suspensions?
 A: The best choice would be something similar to the second (bootstrap) option, but taken an additional step to get an estimate closer to how the model will perform when applied to the underlying population.
Bootstrapping doesn't just give you the ability to estimate the standard error of your performance metric. It also gives you the ability to estimate the bias in the performance metric, arising from potential overfitting of the data set at hand. Under the bootstrap principle, the multiple bootstrap samples from your data set mimic repeatedly taking data sets from the underlying population.
So you train models on multiple bootstrap samples and evaluate each of them both on the corresponding bootstrap sample and on your full data set. Calculate the average bias in your performance metric between the models applied to the corresponding bootstrap samples and to the full data set. That estimates the bias of the metric of your full model, built on your complete data set, when applied to the underlying population. Then you can report an optimism-corrected metric for your model in addition to the original metric.
This optimism bootstrap is implemented in the validate() function of the R rms package for many model types including parametric survival models. It also evaluates several performance metrics beyond the C-index.
Something similar can be accomplished with repeated cross-validation, and cross-validation is an option in the validate() function. It's not clear that provides any advantage over bootstrapping, with its justification noted above.
In terms of number of events versus censored cases, the main issue is that the precision of any estimate will depend upon the total number of events. If there are many predictors relative to the number of events you might have problems with some bootstrapped samples not having enough events to fit your model, but in that case you might already be overfitting.
