1
$\begingroup$

When running a cluster analysis, the algorithm used normally returns a measure of how much variation the clustering explains. e.g.

"This clustering explains 96 % of the variation in the data"

However, I am not sure how much of the variation a clustering method should explain in order to be "good enough", or how little variance explained is basis for rejecting the clustering and trying another approach (e.g. a different amount of clusters).

Any general rules on that?

Thanks!

$\endgroup$
3
  • 3
    $\begingroup$ How something is "good" is determined by you. It depends on the field of the study and the tasks. Clustering - most methods - attempts this or that way to maximize the between-cluster variation (and thus to minimize the within-cluster variation), but how much it succeeds in it depends on the data. $\endgroup$
    – ttnphns
    Aug 26, 2015 at 8:52
  • $\begingroup$ I am aware of that, but I'd still like to know if there is a threshold of some sort, e.g. you reject a clustering if it explains less than 50 per cent of the variation, less than 30 per cent or something else entirely? $\endgroup$
    – SiKiHe
    Aug 26, 2015 at 13:14
  • 1
    $\begingroup$ One interpretation of your demand might be worded as "can a clustering validation criterion tell me if there at all clusters in the data or I should reject clustering results and convince myself that there are no clusters?" Then, for example, look at Gap clustering criterion which, by simulations, tries to answer this question. $\endgroup$
    – ttnphns
    Aug 26, 2015 at 19:05

2 Answers 2

1
$\begingroup$

The comment provided by ttnphns is correct; the percent variation accounted for is so closely tied to the variability of the data that a hard-and-fast rule simply does not exist. Additionally, adding additional clusters will always increase the % variance accounted for, so what you really need to be asking is something along the lines of: is the % increase in variance accounted for worth adding another cluster (increasing complexity).

One approach is to try some number of clusters, calculate the % variance accounted for at each step, and then try to determine some threshold for change in variance beyond which adding clusters had little yield. For example:

set.seed(1)
z = data.frame(x=1:200,y=c(runif(100,0,1),runif(100,1,2)))
p = numeric()

for (i in 1:4){
   K= kmeans(z,i)
   p[i] = 1 - K$tot.withinss / K$totss
}

plot(p,xlab='# of clusters',ylab='% variance accounted',type='o')

Change in % variance accounted for

$\endgroup$
0
$\begingroup$

The clustering where every point is its own cluster explains 100% of the variance, doesn't it?

Is that a useful result? No.

Don't use this measure as a criterion. It's monotone increasing with the number of clusters.

Instead evalute the clustering by your task: does it help solving your actual problem?

In the end, you need a helpful clustering, not one that minimizes/maximizes some mathematical property, don't you? Or is the mathematical property your problem?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.