1
$\begingroup$

I have 2 surrogate indices, and 2 individual markers that were used to compute these indices. I also have demographic variables age, sex, education, smoking, drinking and exercise. The surrogate indices were developed to measure the same health condition. I want to know which of these indices is superior at discriminating mortality, and whether the indices are better than each of the two individual markers at discriminating mortality.

I build 4 models: model 1: Index1 + demographic variables; model 2: Index2 + demographic variables; model 3: marker1 + demographic variables; model 5: marker2 + demographic variables.

I then computed the integrated AUC and Harrell's C-statistic to compare the four models.

The C-statistic and IAUC for model 1 and model 2 were the same (C-statistic=0.76, SE 0.004; IAUC=0.732). For model 3 and model 4, the C-statistic was the same (0.77, SE0.004) while the IAUC for model 1 was 0.70 and that of model 4 was 0.733.

My interpretation of this result was that The two indices had the same discrimination performance, and did not perform better than the basic individual markers from which they were computed.

My questions are:

  1. Is this the best way to answer this research question? Was it proper to just compare C-statistic and IAUC as measures to identify the best index?

  2. Is there a proper way to approach this problem?

$\endgroup$

1 Answer 1

1
$\begingroup$

Concordance probability ($c$-index; generalized AUROC) were not designed for comparing two models. They are just good measures of pure predictive discrimination for a single model. They lack power for comparisons and have a strange distribution. Look at this for powerful and flexible ways to compare models.

$\endgroup$
2
  • $\begingroup$ The link does not seem to work. $\endgroup$ Commented Sep 14, 2023 at 11:06
  • $\begingroup$ Sorry fixed now. $\endgroup$ Commented Sep 14, 2023 at 11:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.