As I understand, AUROC tells us the probability the model will score a randomly chosen positive class higher than a randomly chosen negative class. Meaning that, if AUROC = 0.7, than we expect that for 70% of all new data points positive class will be scored higher, than negative class (am I wrong in this last sentence?).
I'm interested in the particular situation when you have train and test data, and for both you have AUROC = 1, BUT for train dataset the threshold that perfectly separates two classes is 0.5, and for test dataset the threshold that perfectly separates two classes is 0.18. Completely different thresholds, but the same AUROCs. Is this even possible?
My motivation for this question is that I can't see why to check AUROC, if the thresholds that perfectly separates the classes for different data may be different (may be these are two completely different topics and I confuse them?). I mean, if we know AUROC = 1, then we expect our binary classifier to always perfectly separate two classes. But from the single value of AUROC we can't say what threshold to choose so that our classes are perfectly separated. So, AUROC = 1 tells us that our classifier IS ABLE to perfectly separate the data, but we also have to learn what threshold produces this separation. And indeed suppose our classifier always perfectly separates the two classes for any given data, but the threshold producing this separation is always different (first time it's 0.5, next time it's 0.18 and so on). Then how does the value of AUROC = 1 help us, if the threshold can change for different data? The data will always be perfectly separated, but we don't know by which threshold.
My guess is that AUROC estimates expected proportion of new data for which positive classes will be scored higher, than negative classes, and threshold can't change that wildly because we expect the data we use during the training to be representative. But I'm not sure.
I examined this site, but wasn't able to find an answer to this particular question. Can you answer it or just show me the direction where I'm wrong?
EXTRA: some pictures, illustrating my question.
Train dataset with threshold = 0.5:
Test dataset with threshold = 0.18: