2
$\begingroup$

I am using code from Using BIC to estimate the number of k in KMEANS (answer by Prabhath Nanisetty) to find BIC values for K-means using different number of components. However, using iris dataset, I get following results: 

N_clusters        BIC  
1         -863.896405          
2         -674.133038          
3         -616.557809           
4         -603.357368           
5         -582.428798           
6         -596.073710           
7         -590.086212           
8         -579.876476           
9         -554.665433

This is shown in following plot:

enter image description here

The plot after standardization of data:

enter image description here

Is is normal to have negative values for BIC. Which is the best number of clusters by BIC here, especially considering that iris data set has 3 groups? Most negative value in above list is for 1 cluster only.

Improve this question
$\endgroup$
4
  • $\begingroup$ I can't say anything about the program you used. But I can check by "my" program (which is how SPSS does it in two-step cluster analysis). If you are interested, please tell me whether you want the iris variables be standardized before k-means. Note also that BIC does not - as far as I know - check if there is one vs or more-than-one clusters. $\endgroup$
    – ttnphns
    Oct 27, 2018 at 14:30
  • $\begingroup$ I have corrected the link in my question above. It takes you to the code I have used. Also, I have added plot obtained after standardization of data. $\endgroup$
    – rnso
    Oct 27, 2018 at 14:37
  • $\begingroup$ I also use the code from the link you mentioned but I think there's a misunderstanding there. When you calculate the euclidean distance make sure you don't square the result because you want the square root of the sum of squares. I tested with the squared distance and get the same results as you, but if I use the square root I get the expected results $\endgroup$
    – Bernardo Braga
    Mar 6, 2019 at 13:54
  • $\begingroup$ Seems a good point. It will be great if you put this as a full answer, preferably with a figure. $\endgroup$
    – rnso
    Mar 6, 2019 at 14:23

2 Answers 2

Reset to default
2
$\begingroup$

I also use the code from the link you provided.

First thing, it is normal to have negative values of BIC. As you are using BIC = likelihood - penalty you want to find the highest value, which in your first image clearly we would pick N_clusters = 8 and in the second image N_clusters = 9.

I get almost the same if I use the squared euclidean distance:

enter image description here

If I use the euclidean distance I get the expected results and this is the formula I've been using because I've made some tests and it seems correct.

The results using the appropriate euclidean distance gives me this plot:

enter image description here

And here we can obviously see that the appropriate number of clusters to pick is 3 (Setosa, Versicolor and Virginica).

One last note is that it doesn't make sense to set your minimum n_clusters to 1, it should start with 2. I only started with 1 to make the plot look like yours.

Improve this answer
$\endgroup$
1
$\begingroup$

Here is my results. But I used another program with pseudocode explained here.

enter image description here

BIC recommends 3-cluster solution. Clustering was done of standardized iris data variables by k-means (initial centres by RUNFP method).

The scatterplot of the (standardized) data of the 3-cluster partition in the first two principal components (explaining 96% of the data) is below.

enter image description here

Improve this answer
$\endgroup$
2
  • $\begingroup$ These results are clearly very different from mine and they fit with the data very well. Where is the problem in the code that I have used? $\endgroup$
    – rnso
    Oct 27, 2018 at 15:32
  • $\begingroup$ Will you excuse me if I leave digging in the code on yourself? There must be some mistake in it. Note, too, that the correct version of your code needs not give exactly the same results as mine because there exist not one algorithm/formula to calculate BIC for cluster validation. Here is my short overview on clustering validation criteria: stats.stackexchange.com/a/358937/3277 $\endgroup$
    – ttnphns
    Oct 27, 2018 at 15:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.