Detecting outlier points WITHOUT clustering, if we know that the data points form clusters of size $>10$

I'm having a final exam on Data Warehouses and Data Mining (Computer Science bachelor), and the professor recommends some extra exercises on his web-page, as extra material for studying before the exam.

In them, lies the following exercise, among others:

In the following image, we can see three well-defined clusters and two outliers. Describe the application of a technique which will detect the outlier points without performing clustering, however knowing that all [non-outlier] points form clusters of size $>10$. Now, I don't know of any techniques that detect outliers without actually performing clustering, and I looked up the relevant chapters in the teaching book to see if any algorithms or techniques are described. I found a total of one (1!) pages, that briefly states:

To remove outliers, we calculate the sum of distances of a point towards all others. To calculate the distances, we can use Mahalanobis Distance.

After which, the writer of the book (also my class professor) casually lists the following result plot, with no explanation on how it was produced (note that the data in this plot are irrelevant from the exercise above, and are from an earlier book example): And that's it. It doesn't even list a formula or objective function to calculate Mahalanobis distance, which I had to look up on Wikipedia.

Apart from that, the exercise does not mention standard deviation, so the two prominent results I got from stats.stackexchange are of no use (this question and also this one).

I'm frustrated because this is supposed to be Exercise 2, which should mean "easy" and "introductionary", and should require a minimal description and effort to solve, but here I stand stumped.

Does anyone know of any algorithms that do this, relevant to us knowing in-advance that the points form clusters of $size>10$? Any help or pointers towards the right direction would be greatly appreciated. My first thoughts was to describe applying k-means or DBSCAN, but these are actual clustering techniques, and the exercise calls for just outlier detection without clustering.

• Hint: What does the distribution of the nearest-neighbor distance look like? – whuber Jun 9 '18 at 21:17
• @whuber For every object, I could 1) Calculate the distance of it from all others 2) Find the 10 Minimums 3) Store them as outlier_scores. Yes, the two "real" outliers in the graph are going to have huge outlier_scores when compared to every other object, but how can I correctly generalize and say "Yeah their outlier_scores is x times bigger, therefore they're outliers" ? – Dimitris S. Jun 9 '18 at 22:38
• For each object, compute the distance to its nearest neighbor. Draw a picture of the distribution of these numbers: the presence of the outliers will be abundantly clear--and no "clustering" is involved in this process. – whuber Jun 10 '18 at 16:33
• @whuber Thank you. Would I also be correct in thinking that a better pointer of being an outlier or not would be a sum/average of the distances to the 10 closest neighbors, not just one? – Dimitris S. Jun 10 '18 at 20:37
• The problem with that characterization is that "10" was derived from your visual assessment of clustering. The distribution of nearest-neighbor distances identifies outliers regardless of what value you take as the threshold. – whuber Jun 10 '18 at 23:36