I'm interested in calculating the concordance ($c$-statistic) of a proportional hazards model. For reference, the c-statistic is detailed in Stat Med. 1996 Feb 28;15(4):361-87.

The relevant quote from the paper is here:

In predicting the time until death, $c$ is calculated by considering all possible pairs of patients, at least one of whom has died. If the predicted survival time is larger for the patient who lived longer, the predictions for that pair are said to be concordant with the outcomes.

Thus, as far as I can tell, calculating $c$ for a dataset requires predicting a survival time for every observation in that dataset. What survival time is typically used, e.g., median or mean?

Complicating matters, the lifelines module for python implements survival analysis and the $c$-statistic. But, instead of a predicted survival time, it appears to use the predicted partial hazard (i.e., $e^{\beta^T x}$). (The invocation of a $c$-statistic calculator is here, line 151, and the definition of the $c$-statistic calculator is here.) Is using a predicted survival time for the $c$-statistic equivalent to using the partial hazard?


Author of lifelines here. First thing:

But, instead of a predicted survival time, it appears to use the predicted partial hazard (i.e., $\exp \beta^T x$)

I actually use the negative of this quantity (see the line-of-code you pointed out). This'll be important in a moment.

The c-statistic (or concordance index) is invariant under a monotonically-increasing transform. For example, suppose $f$ is monotonically-increasing function (like $\exp$ or $\log$), then the following is true: $$c(x, y) = c(x, f(y)) = c(f(x), y) = c(f(x), f(y))$$

This comes from the definition of the c-index. Heuristically, it's because we only care about ranking values against each other, and not the accuracy of their ratio.

Next note that for the Cox model, the median of the survival curve is actually a monotonically-decreasing function of the linear-predictor, $\beta^T x$ (see below for proof), so we can use either the median or the negative of the linear-predictor (in fact, you've pointed out a small optimisation I can do: I can use the negative of the linear-predictor, and not the negative exponential of it). This isn't generally true for any survival model, but true for the Cox model because of the way the model is defined.

More generally, if you are not using the Cox model, there are more complications than just whether to use the mean or median. For some survival curves, the mean or the median may not exist! You would have to use a higher percentile perhaps.

Proof of above comment:

Suppose, wlog, $\beta^T x > \beta^T y$, then:

$$ e^{\beta^T x} > e^{\beta^Ty}\\ \forall t>0, \;b_0(t)e^{\beta^T x} > b_0(t)e^{\beta^Ty} \\ -b_0(t)e^{\beta^T x} < -b_0(t)e^{\beta^T y} \\ \exp(-b_0(t)e^{\beta^T x}) < \exp(-b_0(t)e^{\beta^Ty}) \\ 0.5 = \exp(-b_0(t_{median_x})e^{\beta^T x}) < \exp(-b_0(t_{median_x})e^{\beta^Ty}) \\ $$

  • 2
    $\begingroup$ Thanks for the clarification. In the code, I would have found it clearer if the name of the argument of concordance_index were something like hazard_scores instead of predicted_event_times. $\endgroup$ – John Kleve Jun 25 '18 at 13:49
  • $\begingroup$ Agree! I've updated the latest code to reflect that, thanks for the suggestion. $\endgroup$ – Cam.Davidson.Pilon Jun 25 '18 at 14:22
  • $\begingroup$ Nice discussion! I also planned to use the concordence index from your lifeline packages, and faced the same question that @John Kleev asked above. $\endgroup$ – SUNDONG Jul 26 '18 at 6:22

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.