# cluster analysis, Ward: how to evaluate number of clusters and their quality?

I have a table of similarities (cosines) and I clustered it with the Ward method. Great outcomes, a wonderful dendogram, but then I tried to evaluate the quality of this cluster solution and I got stuck.

First: identifying the number of clusters in my data (cause in Ward is not like k-means where you have to set a precise number of clusters). I calculated the sum of squares (see attachment) to see how many clusters are there, but there isn't a proper "elbow" in the data, so how many clusters shall I consider?

Second: trying to calculate the purity of the clustering (with the tool CluTo), by indicating 4, 5, 6, 7... clusters, I can see that the purity increases the more clusters I indicate. Of course. If the number of clusters equals the number of instances of my data, then purity is 1 (the maximum). dah.

Any suggestion on how to report this? (number of clusters? quality of the clustering solution?)

• What software are you using? Most software offers some additional statistics that can help make this decision (e.g. the Cubic Clustering Criterion - CCC). – Peter Flom Jul 16 '14 at 11:45
• Great outcomes, a wonderful dendogram Ward's method always gives pleasant dendrograms, even if there is hardly any clusters in the data. – ttnphns Jul 16 '14 at 17:15
• It is better to plot some sort of ratio between the within-sum-of-squares and the between-sum-of-squares. That makes up several clustering validity criterions, such as Calinski-Harabasz, Davies-Bouldin, Gap index, CCC (mentioned by Peter). You may search this site and the web to read about them. – ttnphns Jul 16 '14 at 17:20