Can an Anomaly Detector be Tested with Data that it Labeled?

Question

Is it wrong to leverage a model to label data, then perform a train/test split to evaluate the performance of said model?

Assume I have an unlabeled data set where the missing labels are a binary variable indicating a class and the classes are very unbalanced (typical for anomaly detection scenarios).

I've been thinking about using an anomaly detector to help with the labeling process. The process would look something like this.

1. Assume the data only contain one class
2. Train the model on the entire data set
3. Look closely at the output of the detector and determine if data that ranked as more unique/isolated actually includes the rare class or not.
4. This process might be repeated multiple times with the newly found instances of the rare class removed prior to training the model.

This makes sense to me for labeling, but when it comes to estimating performance this just seems wrong. If I've used the detector to help me label the data, then I'd expect the performance estimates to be overly optimistic because of that. Granted, I (a human) would actually be reviewing and adding labels for the rare class.

Once the data has labels I'd split the data into training and test sets where the training set has no instances of the rare class and the test set has all instances of the rare class as well as many instances of the common class. In such a pipeline, finding "true positives" in the test set seems like a self-fulfilling prophecy. However, the model from this training/test split will be different from the model used to help with labeling since the model parameters will be estimated with less data.

In the context of this conversation, anomaly detector is a generic term and could be one of many different algorithms. For examples see https://builtin.com/machine-learning/anomaly-detection-algorithms.

Answer 1

This "iterating" idea is valid and used in practice for creating datasets. However, if you want an unbiased estimate of the performace of your anomaly detector, you will have to manually label unseen data. If you only check the samples that are positive for an anomaly, you can get an estimate of the precision only. If you also want to know the recall, which is probably relevant, you will have to check enough data to see if there's any anomalies in there. To get a statistically significant sample is then likely problematic. You will need to check ~1000 cases to get a result on a 1 in 100 anomaly.

Answer 2

It doesn't make a lot of sense to evaluate a model vs. labels it (or a very closely related model with different amounts of data or very similar structure) have created. It would indeed have all the biases a particular class of models (or machine learning with the chosen feature representation in general) would have with at least some overlap in training data etc., so would likely be assessed to do really well due to "learning similar things". Generally, one wants to compare to the "truth" or something that is as close to it as one can get (e.g. a committee of expert physicians agreeing on a diagnosis, an extremely good huge model labelling all the data for assessing a much smaller lightweight model that is expected to be much worse but still useful).

However, there are less problematic ways of using imperfect models. E.g. the models can propose labels/mark regions/etc. and then humans can either confirm or manually change the model output. That can substantially accelerate human data labelling without introducing too much of a bias (of course, there's a risk, as people might be more inclined to keep clicking "accept").