I found this tutorial, which suggests that you should run the scale function on features before clustering (I believe that it converts data to z-scores).
I'm wondering whether that is necessary. I'm asking mostly because there's a nice elbow point when I don't scale the data, but it disappears when it's scaled. :)
The issue is what represents a good measure of distance between cases.
If you have two features, one where the differences between cases is large and the other small, are you prepared to have the former as almost the only driver of distance?
So for example if you clustered people on their weights in kilograms and heights in metres, is a 1kg difference as significant as a 1m difference in height? Does it matter that you would get different clusterings on weights in kilograms and heights in centimetres? If your answers are "no" and "yes" respectively then you should probably scale.
On the other hand, if you were clustering Canadian cities based on distances east/west and distances north/south then, although there will typically be much bigger differences east/west, you may be happy just to use unscaled distances in either kilometres or miles (though you might want to adjust degrees of longitude and latitude for the curvature of the earth).
Other answers are correct, but it might help to get an intuitive grasp of the problem by seeing an example. Below, I generate a dataset that has two clear clusters, but the non-clustered dimension is much larger than the clustered dimension (note the different scales on the axes). Clustering on the non-normalised data fails. Clustering on the normalised data works very well.
The same would apply with data clustered in both dimensions, but normalisation would help less. In that case, it might help to do a PCA, then normalise, but that would only help if the clusters are linearly separable and don't overlap in the PCA dimensions. (This example only works so clearly because of the low cluster count)
import numpy as np
import seaborn
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
seaborn.set_theme()
rnorm = np.random.randn
x = rnorm(1000) / 2
y = np.concatenate([rnorm(500) + 5, rnorm(500)]) / 10
fig, axes = plt.subplots(3, 1, sharex=True, sharey=True, figsize=(6, 8))
axes[0].scatter(x, y, s=2)
axes[0].set_title("Data (note different axes scales)")
km = KMeans(2)
clusters = km.fit_predict(np.array([x, y]).T)
axes[1].scatter(x, y, c=clusters, cmap="bwr", s=2)
axes[1].set_title("non-normalised K-means")
def normalise(vals):
"""Normalise values to fit between [0, 1]"""
return (vals - min(vals)) / (max(vals) - min(vals))
xn = normalise(x)
yn = normalise(y)
clusters = km.fit_predict(np.array([xn, yn]).T)
axes[2].scatter(xn, yn, c=clusters, cmap="bwr", s=2)
axes[2].set_title("Normalised K-means")
plt.tight_layout()
It depends on your data.
If you have attributes with a well-defined meaning. Say, latitude and longitude, then you should not scale your data, because this will cause distortion. (K-means might be a bad choice, too - you need something that can handle lat/lon naturally)
If you have mixed numerical data, where each attribute is something entirely different (say, shoe size and weight), has different units attached (lb, tons, m, kg ...) then these values aren't really comparable anyway; z-standardizing them is a best-practise to give equal weight to them.
If you have binary values, discrete attributes or categorial attributes, stay away from k-means. K-means needs to compute means, and the mean value is not meaningful on this kind of data.
Standardization is an important step of Data preprocessing.
it controls the variability of the dataset, it convert data into specific range using a linear transformation which generate good quality clusters and improve the accuracy of clustering algorithms, check out the link below to view its effects on k-means analysis.
https://pdfs.semanticscholar.org/1d35/2dd5f030589ecfe8910ab1cc0dd320bf600d.pdf
As explained in this paper, the k-means minimizes the error function using the Newton algorithm, i.e. a gradient-based optimization algorithm. Normalizing the data improves convergence of such algorithms. See here for some details on it.
The idea is that if different components of data (features) have different scales, then derivatives tend to align along directions with higher variance, which leads to poorer/slower convergence.
Standardization (Z-cscore normalization) is to bring the data to a mean of 0 and std dev of 1. This can be accomplished by (x-xmean)/std dev
Normalization is to bring the data to a scale of [0,1]. This can be accomplished by (x-xmin)/(xmax-xmin).
For algorithms such as clustering, each feature range can differ. Let's say we have income and age. Range of income is [65000,150000] and the range of age [21,90]. Since we calculate the distance(euclidean, manhattan etc), it is important to have the range of each variable to same level.So, I believe to do normalization to bring all the features to a range of [0,1].
