I'm having a final exam on Data Warehouses and Data Mining (Computer Science bachelor), and the professor recommends some extra exercises on his web-page, as extra material for studying before the exam.
In them, lies the following exercise, among others:
In the following image, we can see three well-defined clusters and two outliers. Describe the application of a technique which will detect the outlier points without performing clustering, however knowing that all [non-outlier] points form clusters of size $>10$.
Now, I don't know of any techniques that detect outliers without actually performing clustering, and I looked up the relevant chapters in the teaching book to see if any algorithms or techniques are described. I found a total of one (1!) pages, that briefly states:
To remove outliers, we calculate the sum of distances of a point towards all others. To calculate the distances, we can use Mahalanobis Distance.
After which, the writer of the book (also my class professor) casually lists the following result plot, with no explanation on how it was produced (note that the data in this plot are irrelevant from the exercise above, and are from an earlier book example):
And that's it. It doesn't even list a formula or objective function to calculate Mahalanobis distance, which I had to look up on Wikipedia.
Apart from that, the exercise does not mention standard deviation, so the two prominent results I got from stats.stackexchange
are of no use (this question and also this one).
I'm frustrated because this is supposed to be Exercise 2
, which should mean "easy" and "introductionary", and should require a minimal description and effort to solve, but here I stand stumped.
Does anyone know of any algorithms that do this, relevant to us knowing in-advance that the points form clusters of $size>10$? Any help or pointers towards the right direction would be greatly appreciated. My first thoughts was to describe applying k-means
or DBSCAN
, but these are actual clustering techniques, and the exercise calls for just outlier detection without clustering.
x
times bigger, therefore they're outliers" ? $\endgroup$being an outlier or not
would be a sum/average of the distances to the 10 closest neighbors, not just one? $\endgroup$