Negative BIC in k-means

Question

Probably a simple question but I'm trying to interpret BIC for k-means.

I have some k-means clustering and calculating BIC gives me a negative value, with a plot something like this:

-75000 |                 xxxxxxxxxxx
       |            xxxxx           xxxxx
(BIC)  |        xxxx
       |     xxx
       |   xx
-80000 | x
       ------------------------------------
         2           (k)  25             50

I've searched around but I can't find any results that show a plot like this, apart from on another unanswered question (here).

Does a "smaller" BIC mean that my best number of clusters is "2" (most negative), or "25" (closer to zero), or is my plot just broken?

Answer 1

Generally, the aim is to minimize BIC, so if you are in a negative territory, a negative number that has the largest modulus (deepest down in the negative territory) indicates the preferred model. Hence, in your plot the best case would appear to be "2".

However, the definition of BIC used in the mclust package happens to be the negative of the standard BIC, as the answer by @simone indicates. Therefore, in this package you are looking for the solution with the maximum BIC. In your example, this would be around 25 and above, but below 50.

Answer 2

This may result useful for someone else. I was puzzled with the mclust package because I tried Gaussian mixture models to check whether my data followed a uni- or multi-modal Gaussian distribution. I found that, according to the examples provided in the help, the model that best fitted my data was one with two components (it would suggest the data follows a bimodal Gaussian distribution). However, to my surprise, I found that the model with two components had the highest BIC value (the range of values was on the negative side).

This is because the BIC value calculated in this package is: 2 * loglik - nparams * log(n) instead of the classical: -2 * loglik + nparams * log(n)

This is explained here.