The requirements of the project is to cluster the dataset (using k-means) and then remove the outliers (using MAD) from each of the cluster.

However, I don't feel that it make sense to do that. I think outliers should be removed from the dataset first and then do the clustering.

I'm really new to k-means and machine learning in general. I would really appreciate suggestions. Thanks in advance!

EDIT1: Answering @Tim as to why outliers should be removed:

There are actually 2 process.

  1. running the k-means,

  2. removing the outliers from each cluster

  • 3
    $\begingroup$ Why do you need to remove outliers? $\endgroup$ – Tim Feb 12 '18 at 6:11
  • $\begingroup$ Hi @Tim, it's not really my project, I'm just helping out someone implement his ideas in python and one of the requirements is to remove the outliers after clustering the dataset. But based on his reasoning, outliers are not necessary for the project. $\endgroup$ – Steven Ferrer Feb 12 '18 at 6:15
  • $\begingroup$ Ok, but why? Answering "how" and "if" you should remove them depends on why do you want to do this. $\endgroup$ – Tim Feb 12 '18 at 6:36
  • $\begingroup$ "Based on his reasoning" - so what is his reasoning? Can you ask him to explain it? $\endgroup$ – arboviral Feb 21 '18 at 17:18

K-means can be quite sensitive to outliers.

So if you think you need to remove them, I would rather remove them first, or use an algorithm that is more robust to noise. For example k medians is more robust and very similar to k-means, or you use DBSCAN.

Consider, for example, this one dimensional dataset: 1 2 3 4 101 102 103 104 10000.

  • $\begingroup$ +1 you're right, I tried it last night and the results of k-means is really distorted and inaccurate with the presence of outliers. thanks for the suggestions :) $\endgroup$ – Steven Ferrer Feb 13 '18 at 3:32

I would assume there are two approaches to your task:

  1. Remove outlier first and then apply your clustering algorithm (for this step itself you may use clustering algorithms!). Please note that k-means itself is not a Soft Clustering algorithm so it does not model the overlaps. For that you may use algorithms like Fuzzy C-Means. There you can define an overlap by clusters for which the memberships of a sample are closer than a threshold.

  2. Ignore the outlier removal and just use more robust variations of K-means, e.g. K-medoids or K-Medians, to reduce the effect of outliers.

The last but not the least is to care about the dimensionality of the data. K-Means is not a proper algorithm for high dimensional setting and needs a dimensionality reduction step beforehand.

  • $\begingroup$ +1 i didn't know there is something like k-medians. i should probably try it. thanks for your suggestions :) $\endgroup$ – Steven Ferrer Feb 13 '18 at 3:36

One method for outlier detection is clustering data, and then try to find outliers using median of cluster distances and also number of points in each cluster and like these kinds of methods and measures. Hence, using clustering methods to find outliers is not strange that much and can be a solution to find outliers in some situations.


K-means is an unsupervised algorithm used to find structure in data. Take a simple example: we have the heights and weights of people.

If we run this algorithm as "2- means," the algorithm might find the categories "male" and "female."

enter image description here

But it might not make sense to throw away outliers unless that's really sensible in the problem; just because there is a person who's really fat and really short doesn't mean they're not a person. However, if there's noise in your dataset, that noise should really be thrown away; you only want signal

Please clarify the situation with more details of what the project is.

  • $\begingroup$ Hi @frank, i updated my answer for the clarification. $\endgroup$ – Steven Ferrer Feb 12 '18 at 7:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.