What is the right measure to compare survival models?

Question

Suppose that there are N patients indexed by n = 1, 2, ..., N. Suppose that I estimate two different survival models M1 and M2. Model $i$ gives a risk score $r_n^i$ for patient $n$.

According to https://stats.stackexchange.com/a/49054/22409:

The index of concordance is a "global" index for validating the predictive ability of a survival model. It is the fraction of pairs in your data, where the observation with the higher survival time has the higher probability of survival predicted by your model. As far as I remember it it equivalent to a rank correlation.

Can I use the C-index to say that one model is better than the other? For example, suppose M1 has a C-index of 0.8, and M2 has a C-index of 0.7, does that mean that model M1 is a better predictor of patient survival?

Answer 1

That is in principle what the C-index is intended for. However, note that the C-index does not account for overfitting. Even once you take that into account (e.g. by obtained it using cross-validation or on a hold-out dataset), you would also want to need to consider the uncertainty around the C-index.

Answer 2

If you want to calculate a concordance measure for survival models you could do two things:

1. Calculate the predicted probability ($y$) of survival at a certain timepoint, and then calculate a regular concordance statistic using $1 - y$ and the event (leaving you to decide what to do with censored subjects); OR
2. Fit a survival model and calculate the Rank Correlation for Censored Data. This test is able to take into account censoring, by "computing the c index and the corresponding generalization of Somers' Dxy rank correlation for a censored response variable" (taking from the rcorr.cens function help page in R) (In R, this test is available in the Hmisc package).

IMHO, I'd pick the latter, as the censoring is probably one of the reasons to use a survival model in the first place.

For interpretation: if one is higher than the other, the 'discriminative ability' of one model on that dataset is indeed better (but as the other answers implied, do note that a small difference might not be significant, nor relevant).

Also note that the c-index is not perfect because of multiple reasons. One of them is that the predicted probabilities of outcome might be way off the observed probabilities of outcome, but as long as the ranking of these probabilities show that higher probabilities are associated with the event, discrimination/c-index might be relatively good. That's why often you also check some kind of calibration measure.

Answer 3

I would recommend the Akaike Information Criterion. This measure is derived from the maximum likelihood measures and is a measure of both discrimination and calibration. The Akaike Information Citerion has the favorable property of penalizing for additional variables in the model.

To address whether a higher C-index connotes a better model, the answer is usually "yes" though you would need to provide the 95% confidence interval to address uncertainty around the estimate.

With AIC, a lower value is better with a chi-square distribution for the difference between any two model. One caveat to the AIC is that the models need to be nested. That is, all subjects in model A need to be in Model B.