I've programmed the kmeans++ algorithm in java, it's the regular kmeans but choosing the initial clusters in a smarter way rather than just random. I've made some tests and this seems to be correctly programmed.
But for my purposes I need the computer to decide the K value so another algorithm computes k-means++ with k from 2 to 7 and then choose the one with the maximum value of BIC:
int d = 1; //dimension
double variance = (double) (1.0 / (N - m)) * SSE(dataset,labels,centroids);
double penalty = 0.5 * m * Math.log(N) * (d + 1);
double likelihood = 0;
int i = 0;
while (i < m) {
double ni = grupos[i].length;
likelihood += (ni * Math.log(ni) - ni * Math.log(N) - (ni * 1 *
0.5) * Math.log(2 * Math.PI * var) - (ni - 1) * 1 * 0.5);
i++;
}
return likelihood - penalty; //BIC
The problem is: sometimes (but not purely random) this method gives me different values of K each time I run.
I've tested this method with simple examples which i know the K and it returns the right value of K and the data is correctly clustered.
Is this suppose to vary sometimes with more complex data?