10
$\begingroup$

I read a lot of papers that test k-means with many datasets that are not normally distributed like the iris dataset and get good results. Since, I understand that k-means is for normally distributed data, why is k-means being used for non normally distributed data?

For example, the paper below modified the centroids from k-means based on a normal distribution curve, and tested the algorithm with the iris dataset that is not normally distributed.

nearly all inliers (precisely 99.73%) will have point to-centroid distances within 3 standard deviations (𝜎) from the population mean.

Is there something that I'm not understanding here?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ What if this simply is a quite bad paper? Does not sound like a high class venue to me. $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Sep 2, 2019 at 5:51
  • 1
    $\begingroup$ The claim you quote from the paper is preceded by the assumption that the data is normal. What's needlessly restrictive in that paper is the claim that k-means assumes normality, suggesting that it couldn't be a satisfactory clustering procedure if the data isn't jointly normal. $\endgroup$
    – CloseToC
    Commented Sep 2, 2019 at 7:58
  • $\begingroup$ the paper is published in IEEE $\endgroup$
    – user
    Commented Sep 2, 2019 at 14:11
  • 1
    $\begingroup$ My question related to the experimental of iris data set in the same paper, As I notice that iris data set not normally distributed.kaggle.com/saurabh00007/iriscsv $\endgroup$
    – user
    Commented Sep 2, 2019 at 14:14
  • $\begingroup$ Well, did you check what % of inliers in iris dataset actually lie within 3 s.d. of the centroids? It likely still happens to be true, it just doesn't automatically follow if the distribution isn't normal. Presumably the authors just need to add a one-liner clarifying that. $\endgroup$
    – smci
    Commented Sep 2, 2019 at 21:10

2 Answers 2

Reset to default
13
$\begingroup$

Here is the full quote:

K-means, being an instance of the Gaussian Mixture Model (GMM), assumes Gaussian data distribution [20][26]. It then follows that nearly all inliers (precisely 99.73%) will have point- to-centroid distances within 3 standard deviations ($\sigma$) from the population mean.

It appears in section IV.A.

The application to the Iris dataset, which, as you note, is not normally, distributed, appears in section V ("Experiments").

I do not see a logical problem with first noting an algorithm's properties under certain assumptions, such as normality, and then testing it in cases where the assumption is not valid.

And of course, k-means can be applied to any dataset. Whether it yields useful results is a different matter.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you, assumption of point to mean within standard deviation can be acceptable in a non-normal distributed data set? $\endgroup$
    – user
    Commented Sep 2, 2019 at 14:32
  • $\begingroup$ It depends on the distribution you assume. $\endgroup$
    – Stephan Kolassa
    Commented Sep 2, 2019 at 14:34
  • $\begingroup$ can you explain more?. If I have a skewed right distributed data set, can I add outlier values >mean+4 std and follow the same assumption of paper to detect it?. $\endgroup$
    – user
    Commented Sep 2, 2019 at 14:38
  • $\begingroup$ If you start with a distributional assumption, you cannot just "add outliers". The probability of "outliers" depend on the distribution you are assuming. (What an "outlier" is is also often questionable.) If something about the paper is unclear, it would probably be better to formulate a new question. $\endgroup$
    – Stephan Kolassa
    Commented Sep 2, 2019 at 14:53
  • $\begingroup$ Errors are added to each feature in this paper (5% to 20% of the data set). In case something is not clear about this paper, I will ask another question. Thanks a lot. $\endgroup$
    – user
    Commented Sep 2, 2019 at 15:20
8
$\begingroup$

I'm not sure what the question is exactly, but standard deviation isn't just defined for normal distributions. It's a measure relevant for all data distributions. The farther away you are from the mean (in terms of std) the more unlikely this point is to occur. The only thing special about the normal distribution, regarding the standard deviation is that you know the probability of a point occurring within 1, 2 or 3 standard deviations (e.g. you know that a point has a probability of 99.7% to lie within $\pm 3$ standard deviations from the mean).

This however doesn't mean that standard deviation is irrelevant for other (possibly unknown) distributions. It is still relevant, but you don't know the probability associated with it.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ ok that i mean, but in this paper data set is not normally distributed and still assume data (99.7% to lie within ±3 standard deviations from the mean). my question related to this point $\endgroup$
    – user
    Commented Sep 1, 2019 at 23:52
  • $\begingroup$ I think you're right. This assumption is false, in my opinion. $\endgroup$
    – CaucM
    Commented Sep 2, 2019 at 0:02
  • 7
    $\begingroup$ The farther away you are from the mean (in terms of std) the more unlikely this point is to occur. This might not be true for multimodal distributions. $\endgroup$
    – JAD
    Commented Sep 2, 2019 at 11:54
  • $\begingroup$ You know how likely it is for some event to be occurring within 1, 2 or 3 standard deviations for other distributions as well, so that isn't really special. One special thing is that for a given mean and variance, the normal distribution is the one with the most entropy, so if you only know mean and variance, you'd pick it by the principle of maximum entropy en.wikipedia.org/wiki/Principle_of_maximum_entropy $\endgroup$
    – etarion
    Commented Sep 2, 2019 at 15:21
  • $\begingroup$ This rule can be worked for other distributions? $\endgroup$
    – user
    Commented Sep 3, 2019 at 16:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.