1
$\begingroup$

I have a survival analysis (cox-ph and cox-ph cause specific) that uses a lot of observations (excess of 2million), with many fewer events (around 200,000 in one configuration and 10k in another).

I read online that the c-index is (a) the most popular goodness of fit test of survival analyses, and (b) is optimistic with many censored results. My questions are:

  1. If I do use the c-index, is there a generally acceptable threshold for what is a good value (besides just over .5)?

  2. If I don't use a c-index, what should I use?

$\endgroup$
2
  • 2
    $\begingroup$ Are you building a model to evaluate a hypothesis or for prediction? $\endgroup$
    – Todd D
    May 27, 2022 at 20:50
  • $\begingroup$ Evaluate a hypothesis $\endgroup$
    – tchoup
    May 27, 2022 at 21:20

2 Answers 2

2
$\begingroup$

"Goodness of fit" is a very broad term. The classic book by Therneau and Grambsch devotes at least 4 of its 10 chapters to evaluating different aspects of Cox models: residuals of several types, the functional form modeled for continuous predictors, the proportional-hazards (PH) assumption, and the influence of individual cases on the results.

In your situation with tens of thousands or more of events, you almost certainly will violate the PH assumption and will have to apply your knowledge of the subject matter to determine whether the violation is large enough to matter in practice. Influence of small numbers of outlying cases might not be a big problem, but should be checked.

Frank Harrell, who developed the C-index for survival models, provides several measures of model quality in the validate.cph() function of his rms package:

Statistics validated include the Nagelkerke R^2, Dxy, slope shrinkage, the discrimination index D [(model L.R. chi-square - 1)/L], the unreliability index U = (difference in -2 log likelihood between uncalibrated X beta and X beta with overall slope calibrated to test sample) / L, and the overall quality index Q = D - U. g is the g-index [Gini index] on the log relative hazard (linear predictor) scale. L is -2 log likelihood with beta=0.

Chapter 5 of the second edition of Regression Modeling Strategies goes into detail. The C-index (= 0.5 + (Dxy/2)) is just the fraction of pairs of comparable cases for which the predicted and observed order of events agrees. For a single model that's a useful measure of the ability to discriminate among cases, but it doesn't evaluate overall model calibration, "the ability of the model to make unbiased estimates of outcome." Harrell thinks that the C-index isn't a good way to compare models. The validate.cph() function provides overfitting-adjusted estimates of these measures. The calibrate() function compares predicted and observed probabilities of survival at a specified time of interest.

If you have a specific hypothesis about one or a small number of predictors, then evaluating 2 nested models, one including and one excluding those predictors, is probably the most sensitive test. That can provide an adequacy index to evaluate "the adequacy of the model that ignores the new predictors."

You seem to have some very rich data sets that should provide the opportunity for detailed modeling. Take advantage of that and document the models thoroughly with respect to both discrimination and calibration.

$\endgroup$
1
$\begingroup$

If you are testing a hypothesis, the c-index may not help much. For hypothesis testing, you should select and use the confounders that you believe are most important based on your content knowledge. Then, test whether the covariate of interest is associated with time-to-event in the presence of confounders.

Perhaps you want to compare two or more nested models that contain different confounders and/or confounder specifications? For these comparisons, you can compare with values of the c-index but the Akaike Information Criterion is probably better for comparing nested models.

See: What is the difference in what AIC and c-statistic (AUC) actually measure for model fit?

$\endgroup$
1
  • $\begingroup$ I'm not comparing nested models, unfortunately. I mostly just want to be able to say, yes, this model fits the data, similar to what I'd do with an R^2 in a linear regression. Is that possible / easy? $\endgroup$
    – tchoup
    May 27, 2022 at 23:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.