I'm trying to figure out ROC analysis and have the following question:
Assume there is an test instrument that claims to be able to identify cats based on a series of checks (such as: does the animal have four legs? does it eat mice?...).
I want to test this instrument using ROC analysis and a sample of cats and dogs. More specifically, my hypothesis is the following: "The instrument is better at predicting cats than dogs."
How should I go about testing this with the Area Under the Curve (AUC)?
My initial thought was: I compare the AUC with the cats as positives against the AUC with the dogs as positives. But that would mean that I test one instrument and just switch the positives and negatives around. I would be comparing, for example, AUC 0.65 against 1-0.65 (0.35), which feels intuitively wrong. Or is it not?
EDIT: In the latter way of comparing the AUC, I could stick to comparing the AUC for the cats against 0.5, I believe. As my sample is cats and dogs only, a significant deviation from 0.5 means both cats and dogs AUCs are significantly different. Does that make sense?