# Kmeans Clustering Returning Disproportional Clusters

I am having trouble understanding why Kmeans is returning so many unproportional clusters. For example, here are some of my test results in MATLAB after running my Kmeans algorithm on it:

raw_crime_data = table2array(T(:,start_crime_stats_index:end_crime_stats_index))
k=15 % then I tried k=5
idxk = kmeans(raw_crime_data,k,'Distance','sqeuclidean');
for i=1:k
length(unique(T.city(idxk(:) == i)))
end

k=15
9442
1
2
1
3
1
5
2738
1
6922
2
153
8
24
3
k=5
4299
1
5
10191
8


This issue just keeps happening.

Is it actually an issue?

Shouldn't I have proportional clusters?

Any pro tips on how to use Kmeans in such a way to group these data best?

It is just 10 completely numeric crime patterns.

I have also looked at this post, but it seems to be for text mining with Kmeans which is slightly different than clustering off of purely numeric data.

https://github.com/conradbm/data_science/blob/master/fbi_crime_1980_2014/data_manipulations.m

• Presumably, you have some outliers in your data. I would recommend that you take your k=5 data, find the point that was in a cluster by itself. Look at the distribution of distances from that point to all others. Look at the distribution of distances between all points. I would bet that they are very different distributions. – G5W Jan 17 '17 at 22:46
• This is true, the distributions are incredibly different. So eventually, without throwing out data, I just have to settle on a K that is good enough? Whats a good decision point on that? – bmc Jan 17 '17 at 23:15
• There is no clear answer to this. There is some guidance available. See This Cross Validated Post and This Stack Overflow Post. But ultimately, it is a choice you make depending on your problem. Both your k=5 and k=15 clusterings have a group with 8 points. Is that rubbish or a micro-cluster? You have to look at your data and your questions to decide. – G5W Jan 18 '17 at 0:17

## 1 Answer

I had a similar issue on another platform (PySpark), and I think it's independent of the coding because it's a data issue. Try reframing the data either with logical quantiles, or by removing high-leverage points (One approach using linear models).

The concept behind this - if your data are crime rates by city, a city with a very high auto theft rate may form its own cluster, and the KMeans algorithm, in trying to maximize distance between clusters while minimizing distance between same-cluster points, will be reluctant to split up a much larger cluster with less variation. The high-leverage points can be pushed into their own cluster, allowing the algorithm to try to split up the much larger cluster.

I've personally seen Quantiling take a KMeans solution in which one of the factors dominates and turn it into a solution in which all factors are valid. The quantiles artificially standardize variation by reducing the number of possible values that a cluster can be fit upon. Do you know which of your features has the highest variance or seems to be driving most of the cluster solutions?