0
$\begingroup$

I am currently working on a dataset with 14 continuous features, a categorical target over five classes, and 90,000 samples. My current goal is to explore outliers in the dataset, and to that end I have been experimenting with Scikit-Learn's implementations of Isolation Forest, Local Outlier Factor, and One Class SVM. Here is a summary of the number of outliers detected and the two most-outlying classes for each model:

Local Outlier Factor identifies 830 outliers (44% in class 3 and 29% in class 0)

Isolation Forest identifies 16,430 outliers (52% in class 3 and 25% in class 2)

One Class SVM identifies 45,000 outliers (34% in class 3 and 31% in class 0)

I might also add that Local Outlier Factor and One Class SVM's third most-outlying class is also class 2. But to the point, I only have a rudimentary understanding of each of these algorithms and overall limited experience in outlier detection, so I am having a hard time interpreting these results. Moreover, it seems highly odd that the one class SVM would decide that exactly half my dataset is outlying, and I would like to understand why class 3 consistently makes up the largest proportion of the set of outliers.

If anyone could point me towards some analytic techniques or explanations which might help me better analyze these results, I would be very appreciative.

$\endgroup$
8
  • 1
    $\begingroup$ Are these the results on your training set or testing/validation set? $\endgroup$
    – Jon Nordby
    Commented Jan 21, 2021 at 22:04
  • $\begingroup$ @jonnor these are the results on all my data, I don't have a labeled set of outliers to use. $\endgroup$ Commented Jan 22, 2021 at 14:11
  • $\begingroup$ You should still do train/val/test splitting, to have test results on indepdentent samples from those used during fitting and hyperparameter optimization. $\endgroup$
    – Jon Nordby
    Commented Jan 23, 2021 at 7:58
  • $\begingroup$ What do you mean by 'targets' - that sounds like you have a label/ground truth? $\endgroup$
    – Jon Nordby
    Commented Jan 23, 2021 at 8:00
  • $\begingroup$ Are you doing one outlier detection model per 'target'? Or a joint model? $\endgroup$
    – Jon Nordby
    Commented Jan 23, 2021 at 8:01

1 Answer 1

2
$\begingroup$
  • All of those methods have hyperparameters, are you sure you use reasonable values for the hyperparameters? Scikit-learn has nice article on them.
  • Not much can be said without access to your data and actual results. What you could do, is to just produce a lot of all kind of plots and summary statistics for all of the parameters, grouped by the classification results to try to understand what is going on. It is useful to look at the raw predictions as well, you can just take something like ten least anomalous and ten most anomalous samples (as classified by the algorithm) and compare them, how do they differ?
  • You can check the Interpretable Machine Learning book by Christoph Molnar that is freely available online for more ideas and description of most advanced methods for interpreting the results.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.