I have 200k data samples which are class imblanced (10% positive class, 90% negative class). I split the data in exactly half so my training set is 100k samples and my test set os 100k samples. I train algorithms A and B.
Algorithm A discriminates between the two classes using the test set and it achieves an AUC of AUC_A. Algorithm B, which is an improvement on A, gets AUC we call AUC_B. I want to determine if AUC_B > AUC_A by chance or not (statistically significant).
What is an algorithm to determine this answer? (say we set p<0.001)