What are the best (recommended) pre-processing steps before performing k-means?


2 Answers 2


If your variables are of incomparable units (e.g. height in cm and weight in kg) then you should standardize variables, of course. Even if variables are of the same units but show quite different variances it is still a good idea to standardize before K-means. You see, K-means clustering is "isotropic" in all directions of space and therefore tends to produce more or less round (rather than elongated) clusters. In this situation leaving variances unequal is equivalent to putting more weight on variables with smaller variance, so clusters will tend to be separated along variables with greater variance.

enter image description here

A different thing also worth to remind is that K-means clustering results are potentially sensitive to the order of objects in the data set$^1$. A justified practice would be to run the analysis several times, randomizing objects order; then average the cluster centres of the correpondent/same clusters between those runs$^2$ and input the centres as initial ones for one final run of the analysis.

Here is some general reasoning about the issue of standardizing features in cluster or other multivariate analysis.

$^1$ Specifically, (1) some methods of centres initialization are sensitive to case order; (2) even when the initialization method isn't sensitive, results might depend sometimes on the order the initial centres are introduced to the program by (in particular, when there are tied, equal distances within data); (3) so-called running means version of k-means algorithm is naturaly sensitive to case order (in this version - which is not often used apart from maybe online clustering - recalculation of centroids take place after each individual case is re-asssigned to another cluster).

$^2$ In practice, which clusters from different runs correspond - is often immediately seen by their relative closeness. When not easily seen, correspondence can be established by a hierarchical clustering done among the centres or by a matching algorithm such as Hungarian. But, to remark, if the correspondence is so vague that it almost vanishes, then the data either had no cluster structure detectable by K-means, or K is very wrong.

  • 3
    $\begingroup$ The randomizing, re-running, averaging and final run is a very good advice. Thanks $\endgroup$ Commented Jan 30, 2012 at 16:08
  • 1
    $\begingroup$ How would k-means be sensitive to ordering? $\endgroup$
    – SmallChess
    Commented Feb 23, 2017 at 5:39
  • 1
    $\begingroup$ @StudentT, I've added a footnote for that. Thank you. $\endgroup$
    – ttnphns
    Commented Feb 23, 2017 at 7:19
  • 1
    $\begingroup$ @ttnphns how do determine quantitatively that variables have "quite different variances"? $\endgroup$ Commented Oct 18, 2018 at 12:09
  • 1
    $\begingroup$ @camillejr, please start by checking this Q: stats.stackexchange.com/q/418427/3277. $\endgroup$
    – ttnphns
    Commented Nov 6, 2019 at 18:06

Depends on your data I guess. If you would like trends in your data to cluster together regardless of the magnitude, you should center. eg. say you have some gene expression profile, and want to see trends in gene expression, then without mean centering, your low expression genes will cluster together and away from high expression genes, regardless of trends. Centering makes genes (both high and low expressed) with like expression patterns cluster together.

  • $\begingroup$ I am actually comparing different features that have their own scale. For instance I am comparing GC content which has a range from about 0.3 to 0.5 which may seem to be small but the difference is quite important; some other features have broader ranges, some others range in very small scales. $\endgroup$ Commented Jan 17, 2012 at 10:21
  • $\begingroup$ So, you are clustering different factors? Could maybe use some weight or transformation of values. $\endgroup$ Commented Jan 17, 2012 at 10:45
  • $\begingroup$ No, I am comparing all continuous variables $\endgroup$ Commented Jan 30, 2012 at 16:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.