19
$\begingroup$

I've learned that when choosing a number of clusters, you should look for an elbow point for different values of K. I've plotted the values of withinss for values of k from 1 to 10, but I'm not seeing a clear elbow. What do you do in a case like this?

Troublesome KMeans

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ There exist many clustering criterions, "SS elbow" rule being just one and not the best. Try other. It is as well likely that you don't have clusters in your data. $\endgroup$
    – ttnphns
    Commented Mar 12, 2014 at 6:30
  • $\begingroup$ @ttnphns What is this mystical other you speak of? How can I not have clusters in my data? How do I know? $\endgroup$
    – Glen
    Commented Mar 13, 2014 at 16:39

4 Answers 4

Reset to default
9
$\begingroup$

Wrong method?

Maybe you are using the wrong algorithm for your problem.

Wrong preprocessing?

K-means is highly sensitive to preprocessing. If one attribute is on a much larger scale than the others, it will dominate the output. Your output will then be effectively 1-dimensional

Visualize results

Whatever you do, you need to validate your results by something other than starting at a number such as SSQ. Instead, consider visualization.

Visualization may also tell you that maybe there is only a single cluster in your data.

Improve this answer
$\endgroup$
2
  • $\begingroup$ What are some good visualization options for multi-dimensional data? $\endgroup$
    – Jeremy
    Commented Mar 14, 2014 at 14:44
  • 1
    $\begingroup$ Depends on your data. Some data can be projected well, because it has much lower intrinsic dimensionality. Time series can easily be plotted, and if your data is a serialized image, visualize it as images? By any means, visualization depends on your data, there won't ever be a one-size-fits-all solution. $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Mar 15, 2014 at 2:49
5
$\begingroup$

One way is to manually inspect the members in your clusters for a specific k to see if the groupings make sense (are they distinguishable?). This can be done via contingency tables and conditional means. Do this for a variety of k's and you can determine what value is appropriate.

A less subjective way is to use the Silhouette Value:

https://stackoverflow.com/questions/18285434/how-do-i-choose-k-when-using-k-means-clustering-with-silhouette-function

This can be computed with your favorite software package. From the link:

This method just compares the intra-group similarity to closest group similarity. If any data member average distance to other members of the same cluster is higher than average distance to some other cluster members, then this value is negative and clustering is not successful. On the other hand, silhuette values close to 1 indicates a successful clustering operation. 0.5 is not an exact measure for clustering.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Glen, I personally think your answer is incomplete. The 1st paragraph looks unclear. What is that "manual inspecting", can you describe the procedure please? Then, Silhouette is "less subjective" than what? And why? $\endgroup$
    – ttnphns
    Commented Mar 12, 2014 at 9:38
  • $\begingroup$ @ttnphns answer updated. $\endgroup$
    – Glen
    Commented Mar 12, 2014 at 16:21
  • $\begingroup$ contingency tables and conditional means This is further mystical. What should I do with them do arrive "subjectively" at a good k? $\endgroup$
    – ttnphns
    Commented Mar 13, 2014 at 5:55
  • $\begingroup$ @ttnphns If the poster has a question about it I will follow up. As I said you should check to see if the groupings are distinguishable. It seems clear to me. $\endgroup$
    – Glen
    Commented Mar 13, 2014 at 16:39
  • $\begingroup$ So if I get low silhouette values (~.35) it might indicate that this data doesn't really have good clusters? $\endgroup$
    – Jeremy
    Commented Mar 18, 2014 at 14:54
0
$\begingroup$
  • No elbow in for K-means does not mean that there are no clusters in the data;
  • No elbow means that the algorithm used cannot separate clusters; (think about K-means for concentric circles, vs DBSCAN)

Generally, you may consider:

  • tune your algorithm;
  • use another algorithm;
  • do data preprocessing.
Improve this answer
$\endgroup$
-1
$\begingroup$

We can use the NbClust package to find the most optimal value of k. It provides 30 indices for determining the number of clusters and proposes the best result.

NbClust(data=df, distance ="euclidean", min.nc=2, max.nc=15, method ="kmeans", index="all")

Improve this answer
$\endgroup$
1
  • $\begingroup$ Welcome to the site! Could you expand on this answer? While helpful, a little more detail would make it more useful. $\endgroup$
    – mkt
    Commented Sep 25, 2018 at 11:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.