# kmeans results selection

I have a one dimension weather temperature data and I need to split into 7 clusters(maybe less). Temperature data in the very low end and high end is not as common as middle range values. So using kmeans might violate its assumption that each cluster need to be relative equal sized. But hierarchy clustering dose not give a good clustering result in this case. I still tried kmeans. The results seems relative random, how decide which result is better than the other?

   #use the default value,converged after 2 runs
y_3<-kmeans(y,7)

(between_SS / total_SS =  95.6 %)
cluster centers      size
2      52.50034   37
1      57.12902  122
6      60.65326  238
7      64.25651  270
5      67.89233  241
4      72.29133  154
3      78.10350   78

#try to increase the iteration but it converged after 2 runs

y_4 = kmeans(y,centers =7, iter.max = 1000)
(between_SS / total_SS =  95.9 %)
cluster centers      size
1       54.72145   97
3       59.56208  226
5       63.10763  249
6       66.42057  241
4       70.04897  176
2       74.32394  104
7       79.47868   47

#tried to increase iteration, also set the initial randomized set as 10 , converged after 3 runs

y_6 = kmeans(y,centers =7, nstart=10, iter.max = 100)
(between_SS / total_SS =  95.9 %)
cluster centers      size
3       54.72145   97
5       59.56993  227
7       63.13466  251
4       66.46375  241
1       70.10488  175
2       74.53701  109
6       79.90417   40

• Why use k-means for one-variable data? Why won't simply binning by eye on a plot be helpful? – ttnphns Mar 25 '17 at 12:05
• Can you give us a sense of what the data is like? It might help to remove outliers and then do kmeans. – roundsquare Mar 25 '17 at 15:58
• Please add a histogram plot. – Has QUIT--Anony-Mousse Mar 29 '17 at 20:16

I am all but a fan of k-means. Try to visualize your data, and you may understand why...

Your data apparently is one-dimensional. Then you have much nicer techniques available than k-means...

Do Kernel Density Estimation. Maxima in the estimate are your clusters, while they should be separated at minima.

Plot a histogram of your data, and the KDE. If you can't see clusters in these plots, the algorithms won't find any either...

Did you plot the distribution of this variable? K-means is very unstable, and affect by outliers. The centers number can choose first using other methods, such as hierarchical clustering. For one dimension, you can try histogram plot to clustering or decide the cluster numbers.

Two things to do when using kmeans:

• find optimal k number of clusters using both the elbow method and CH index;
• use a cluster bootstrap to check the stability of each cluster.

The approach is well documented in the book Practical Data Science With R. You are lucky because the chapter on unsupervised learning is fully available free of charge as a sample.

That said, I find it a little overshot to use clustering for a single dimension. Wouldn't a manual empirical analysis of the distribution accomplish the objective already?