I'm doing some anomaly detection, basically classifying stuff as normal (0) or aberrant (1)
As with all anomalies, they are rare, so during the train/test phase it's not good enough for me to just look at general training error (which would be reduced to 10% if I just guessed class 0 (normal) all the time!)
with 90% non-aberrant data, I've decided to use A/B as my metric for training accuracy
where A = number of false negatives (saying it's normal when it's actually aberrant)
and B = # of positives (both false, and real)
So now, when I make a prediction, I also output that value (actually the average of that value when I rerun the test 15 times, and give each run an unweighted vote). And then I can say "oh, my test value is small enough, I can be confident in the prediction"
My question is whether or not this is sound practice, and if there are any better metrics for anomaly detection?