I am comparing ROC AUC values for different feature selection methods. Some methods are deterministic and output the same AUC score every time and some methods are non-deterministic and output different scores every time. I would like to compare the non-deterministic methods with the deterministic ones.
Say, I have an array of AUC scores from a non-deterministic method (e.g. [0.6, 0.65, 0.57, 0.71]) and I want to compare it to the AUC score from a deterministic method (e.g. 0.72).
I need to find out i) whether the two methods perform significantly different (e.g. at a 95% confidence level) and ii) if yes, which method performs better.
What statistical test(s) can you suggest to answer above questions?
Thank you!
