0
$\begingroup$

I want to perform semi-supervised anomaly (novelty) detection on data using machine learning methods (e.g. one class SVM). Is it sensible that in pre-processing step, I use outlier detection techniques (such as isolaton forests) to cleanse normal data? By semi-supervised anomaly detection I mean we only have limited data with normal label but most of the data does not has label.

$\endgroup$
0
$\begingroup$

Yes.

But first, you need to create a suitable evaluation data set. If you lack sufficient labeled data (a common problem in anomaly detection), you can generate synthetic test data by taking normal data that appears anomaly-free and injecting random anomalies of the kind(s) you wish to detect. You can perform other data augmentation (scaling, shifting, etc) to generate arbitrarily large data sets, and then use these for evaluation.

Once you have a good enough evaluation set, the real answer to your question is that the method that performs best is the best method. If performing outlier detection before-hand improves the evaluation metrics, then you should do it :)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.