I have been learning about the use of machine learning algorithms and their application to particle physics.
Now, I have some doubts concerning what to do with the results. Let me explain: imagine that we have two theoretical models to explain the data. I go through all the process of setting up the discriminating variables, choose the method, train the classifier and test it against overfitting, the works.
Consider then that at the end of the process I have a classifier that has an area under the ROC curve of, say, $0.7$. If I understand it correctly, that means that from a set of $N$ events, $70\%$ will be correctly classified (is this right?)
My question is then, what to do with this classifier, or as a matter of fact, how can we use a statistical model, with some non null probability of misclassification in order to infer which theoretical model is the one that most likely explains the experimental data?
The question seems like something that should be in any statistical book but I can't seem to find any reference that explains this issue.