What is the most robust way to compare 2 Cox regression models? Comparison of HRs, C-index?

Question

I am a medical doctor not a statistician. I have 60 patients who were assessed by 400 doctors (each doctor individually evaluated the clinical data on each patient). The doctor was then asked to say whether the patient had disease A or not. I have the survival time and vital status (dead or alive) on each patient.

There is no diagnostic reference standard for this disease, but some have hypothesised that outcome (or mortality) may be a reasonable way of validating diagnostic accuracy (this is not in question - it has precedence in the literature). Therefore we are using prognostic accuracy as an indication of a doctor's ability to accurately diagnose the disease.

I am evaluating the prognostic significance of each doctor's diagnosis using Cox regression, and I also calculated the c-index for each doctor. For each doctor, I, therefore, have a survival model e.g.

c index = .72748 Hazards ratio = 3.0436 p =.0117 95% CI = 1.2815-7.2288

(a total of 400 such models)

I want to compare them, to see whose categorization of "have disease A or not" is the most prognostic significant. I first thought that this could be simply done by comparing the c-index for each doctor but having looked at previous posts, I suspect this is flawed. See for example

How to compare Harrell C-index from different models in survival analysis?

So, the question is, what is the best way to compare the prognostic significance of two survival models?

Answer 1

Your application is different from that in the thread that you linked, so the issues are different.

In the linked thread, the issue is comparing different forms of models, containing for example different complements of predictor variables. In your case you evidently have the same underlying model in all cases, but with the predictor variables scored by different doctors. The form of all the models is the same, and your question is which doctors do the best job of scoring the predictor variables in a way that relates best to outcome.

If I understand this correctly, then the C-index would be suitable for your application, even if it is not the best for comparing different forms of models. For each regression the C-index is equivalent to the fraction of pairs of event times that were predicted in the correct order by the model, given that particular doctor's scoring of the predictor variables. So it provides a straightforward measure of doctor skill that is also fairly easy to explain.

That said, I would not put too much weight on any one particular examination of "whose categorization ... is the most prognostic significant." This is likely to be very dependent on the particular sample of 60 cases that you have assembled. It might be wise to repeat your analysis on multiple bootstrap samples of the data to evaluate this possibility.