0
$\begingroup$

I am using the ABOD method to identify multi-dimensional outliers in my data. Everything is done in R as I am not familiar with other languages for such purposes.

I use a function that calculates the outlier degree and gives me a vector that contains the scores of all my observations. In theory any observation with a low ABOD is an outlier and any observation with a high ABOD is not an outlier.

I am struggling with determining the interval that I should use as my filter. Do you have any suggestions?

Note: I have thought about using a robust version of Tukey's method for detecting outliers (i will apply it on the ABOD scores and see if there are any extremes according to it). Is that a sensible solution to my problem?

My question is strictly theoretical: I do not want a practical solution for it as CV is not the place for that. I only want to check if I am doing something sensible and if it can have useful results.

$\endgroup$
  • 1
    $\begingroup$ The definition of an "outlier" will vary depending on the application. Since your question is "theoretical", my suggestion would be to create some benchmark problems where you can simulate data with known outliers vs. inliers. Then you can test various formulations of your outlier detection (and do training/cross-validation of your hyperparameters). The benchmark-data would depend on your target application, of course. (Perhaps this reference may be relevant, though I have not read it.) $\endgroup$ – GeoMatt22 Dec 6 '16 at 16:11
  • 1
    $\begingroup$ "I do not want a practical solution for it as CV is not the place for that." I (most of us) love practical solutions. If you mean that you are not asking for code, fair enough; otherwise this comment is curious! $\endgroup$ – Nick Cox Dec 6 '16 at 17:08
  • $\begingroup$ Keep in mind that multivariate outliers have an added issue; the direction to define an outlier. $\endgroup$ – Michael Chernick Dec 6 '16 at 23:46
  • $\begingroup$ I used the influence function for bivariate correlation in some of my work at ORNL in the late 1970s. That defines a direction to use when you are concerned with using data to estimate bivariate correlation. This is given in my published paper in the American Journal of Mathematical and Management Sciences in the early 1980s. My work was motivated by an unpublished paper of Colin Mallows and Gnanadesikan's multivariate book. $\endgroup$ – Michael Chernick Dec 6 '16 at 23:56
  • $\begingroup$ @Nick Cox Well i mean that i am not asking for code and because people on this website like to assume things and my questions get flagged frequently for various reasons including "you should post your question elsewhere". $\endgroup$ – Emil Filipov Dec 7 '16 at 7:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.