1
$\begingroup$

Different comorbidity indices are widely used tools for describing patients' comorbidity statuses. A large score = many illnesses, and vice versa. I would like to compare two such indices to examine which better predicts one-year mortality.

Studies have used different estimates to describe "discriminative ability". Which of them should I choose for my goal? Plotting a ROC and calculating AUCs for both indices? Recommendations for R packages are also very welcome. Finally, are there any good Bayesian ways?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Plotting the ROC will not help in any way. Computing AUROC (aka c-index; c=concordance probability) helps as a secondary measure but not as a primary measure because of its insensitivity---it doesn't properly reward extreme predictions that are correct. The gold standard is the log likelihood and quantities derived from it such as pseudo $R^2$. Another excellent metric is the variance of the predicted values (here on the P(death) scale). I deal with these in detail at fharrell.com/post/addvalue.

Make sure you take a look at the Elixhauser and Schweiss comorbidity indexes and avoid the Charlson index, which contains an arithmetic error.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ I’ve seen you post a couple of times that AUC is nice but not quite the gold standard. Could you please expand on this? The way I’ve read this is that AUC is a nice absolute gauge of performance in the sense that we know $0.95$ is usually really good and $0.55$ is pretty bad, but it’s not quite the right metric for comparing models, only to decide if one model on its own has solid predictive power. $\endgroup$
    – Dave
    Commented Feb 10, 2021 at 13:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.