# Interpretation of Cluster Distortion on Normalized data

I have a clustering problem which I solved using KMeans clustering. I also know that the Elbow Method for cluster evaluation can be used to approximate a feasible pick for the number of clusters.

I see alot of individuals using the cluster inertia for their Elbow Method plot. I also discovered that some like to use the 'Distortion' which should correspond to the average euclidean distance between points in some cluster to its centroid. This can be achieved with something like

distortion = sum(np.min(cdist(X, kmeanModel.cluster_centers_, 'euclidean'), axis=1)) / X.shape[0]


I personally find the distortion metric more intuitive for such an evaluation.

Note that my data is normalized as $$(x - \mu)/\sigma$$, which aims to make the underlying data roughly normal distributed.

How should I interpret the distortion? My intuition is the following: when I see an average distortion on $$k=9$$ clusters to be $$1.45$$, does this now mean that I can expect the average datapoint in the average cluster to be within $$1.45$$ standard deviations of the cluster centroid?

According to this post, the distortion for $$k$$ centroids $$\vec{\mathrm{m}} = [\mathrm{m}_1, \dots, \mathrm{m}_k]$$ and class assignements $$\vec{c} = [c_1, \dots, c_n]$$ (where $$c_i$$ is the class found for point $$\mathrm{x}_i$$) obtained from running k-means on data $$\mathcal{D} = \{\mathrm{x}_1, \dots, \mathrm{x}_n\}$$ is defined as $$J(\vec{c},\vec{\mu}) = \sum_{i=1}^n\lVert \mathrm{x}_i - \mathrm{m}_{c_i}\rVert^2$$ So distortion is not the average, however if you multiply it by $$1/n$$ you do indeed get the average squared distance of a random point to its assigned centroid. Using your values, $$\frac{1}{n}J(\vec{c},\vec{\mu}) = 1.45$$ tells you that
On average a point is inside a ball of radius $$\sqrt{1.45}$$ around its centroid
However since your data is normalized, the un-normalized data will we inside an ellipse as the ball is transformed differently along each dimensions by the mapping $$\mathrm{x} \mapsto \sigma\odot\mathrm{x} + \mu$$.
• Well then you normalize each dimension separately, so each dimension $i$ has a different mean $\mu_i$ and std $\sigma_i$. Being in a ball of radius $r$ for the normalized data along dimension $i$ means being in an ellipse of radius $\sigma_i r$ for the transformed data. So yes it's an average deviation of $1.45\sigma_i$ in your case, but the deviation depends on which dimension you're considering. Commented Feb 23, 2021 at 13:31