My answer is not limit to K means, but check if we have curse of dimensionality for any distance based methods. K-means is based on a distance measure (for example, Euclidean distance)
Before run the algorithm, we can check the distance metric distribution, i.e., all distance metrics for all pairs in of data. If you have $N$ data points, you should have $0.5\cdot N\cdot(N-1)$ distance metrics. If the data is too large, we can check a sample of that.
If we have the curse of dimensionality problem, what you will see, is that these values are very close to each other. This seems very counter-intuitive, because it means every one is close or far away from every one and distance measure is basically useless.
Here is some simulation to show you such counter-intuitive results. If all of the features are uniformly distributed, and if there are have too many dimensions, every distance metrics should be close to $\frac 1 6$, which comes from $\int_{x_i=0}^1\int_{x_j=0}^1 (x_i-x_j)^2 dx_i dx_j$. Feel free to change the uniform distribution to other distributions. For example, if we change to normal distribution (change runif
to rnorm
), it will converge to another number with large number dimensions.
Here is the simulation for dimension from 1 to 500, the features are uniform distribution from 0 to 1.
plot(0, type="n",xlim=c(0,0.5),ylim=c(0,50))
abline(v=1/6,lty=2,col=2)
grid()
n_data=1e3
for (p in c(1:5,10,15,20,25,50,100,250,500)){
x=matrix(runif(n_data*p),ncol=p)
all_dist=as.vector(dist(x))^2/p
lines(density(all_dist))
}