# Inputs to k-means are often normalized per-feature. Why not fully whiten the data instead?

We often normalize inputs to the k-means algorithm by 1) subtracting the mean on a per-feature basis and 2) dividing by the standard deviation on a per-feature basis. Some rational behind this is discussed here:

Are mean normalization and feature scaling needed for k-means clustering?

But it seems strange to assume that the features aren't correlated, so my question is, why don't we fully whiten the data instead? In other words, if the data has mean $\mu$ and covariance $\Sigma$, why not preprocess each sample using $\tilde{x} = \Sigma^{-1/2} (x - \mu)$ ?

The only argument I see is that this will get computationally difficult when the dimensionality of $x$ gets very large, for example $\gg 100$, but are there other reasons?

Thank you.

• It is possible. You may pre-process data in any way if it has sense relative to the future result obtained. But - I'd notice - K-means doesn't assume variables to be uncorrellated. What it assumes is that clusters within the data are, roughly, spherical, and it is another thing. – ttnphns Jun 24 '14 at 11:05
• Yes, but the data will be "more spherical" after full whitening than after per-feature normalization. – rd11 Jun 24 '14 at 11:14
• The whole data cloud - yes. But not necessarily clusters in it. Normalizations (any) are not required by K-means, but they may have sense and often do have. – ttnphns Jun 24 '14 at 11:15
• Agreed! Although now we're getting into a different topic :). You could say the same about per-feature normalization. – rd11 Jun 24 '14 at 11:17