I have a dataset with two classes of elements. I also have two methods which assign (complementary) probabilities to each element in the dataset of belonging to either class.
Given that I work with probabilities (instead of hard 0,1 classification values), I was pointed to scoring rules as a way to asses which method performs better. The two most used rules appear to be:
- Logarihmic scoring rule (Log loss, logistic loss, cross-entropy loss)
- Brier/quadratic scoring rule (Brier score)
with Log loss apparently being the standard approach (is it?). I also found
roc_auc_score, an implementation of the:
- Area Under the Curve (AUC, ROC-AUC)
which appears to do pretty much the same thing.
My question is: is either one of these inherently "better" than the other in some form? I also could just use all three. Is this advisable?