2
$\begingroup$

I have a general question about ROC curves and how adding predictors affect AUC values.

  • Let's say I have a model that contains only predictor A and produces an AUC of 0.6.
  • I then add into the model a predictor B that produces an AUC 0.7.
  • I further then add a predictor C into the model (A + B + C) and an AUC of 0.8 is produced.

My question is, if I only had a model of A + C, can it be assumed that the AUC will be less than 0.7 (eg, the model with A + B)?

Or is it possible that the model of A + C could have generated a AUC of say 0.75? If this is true, could it be suggested that predictor variable B is less valuable than variable C in prediction modelling?

$\endgroup$
1
  • $\begingroup$ Can you specify the time of model you're using? Answering this question will depend entirely on what model and what type of data you have. $\endgroup$
    – Calimo
    Mar 17, 2020 at 10:59

1 Answer 1

1
$\begingroup$

If this is true, could it be suggested that predictor variable B is less valuable than variable C in prediction modelling?

AUC of a ROC curve might not be the best way to check which predictor variable is more or less valuable. The curve is based on ranking instances. Adding a variable that slightly changes the probabilities of a model could result in a completely different ranking.

My question is, if I only had a model of A + C, can it be assumed that the AUC will be less than 0.7 (eg, the model with A + B)? Or is it possible that the model of A + C could have generated a AUC of say 0.75?

Generally speaking, it's harder to improve an AUC from 0.7 to 0.8 than from 0.6 to 0.7. The information that B and C share might affect the results as well.

Apart from the fact that AUC is not a proper metric for this argument, you can suggest that C contains more information for the given task. Adding C probably leads to 0.70+ not 0.70- AUC (e.g. mutual information in B and C).

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.