Combine two k-means models for better results

Question

I am clustering some pretty fuzzy data with a special k-means like algorithm (a change of algorithm is not an option). Due to random initialization of cluster centers and the fuzziness of the data the results of two identically trained models are different.

My colleague asked if I could combine the results of the models to get a better model. He suggested to keep clusters where both models agree on (to a certain %) and leave out clusters that are (too) different in both models with the reasoning, that data in these clusters is not really clustered correctly. To assign new data you would let it run on both models and if they both agree to assign the new data to a valid, agreed-on cluster, you have the clustered result. If not, then the solution is that the new data can´t be clustered properly.

This idea sounds pretty unintuitive to me but I couldn´t come up with a proper argument against it so far. Of course the new, combined clustering method would "loose" some data that is labelled as "can´t be clustered", but this is acceptable.

Is my intuition right that this will not really work or is this really a valid approach and I was just unsuccessful with my research and haven´t heard of something like that yet?

Answer 1

Perhaps someone with more background in this area will chime in but I can tell you at least that the idea your colleague presented falls into a valid field of study called consensus clustering or ensemble clustering. The particular implementation he suggests, as far as I can tell, is not an existing method. There are, however, methods that will combine multiple clustering results, fuzzy or hard, to determine optimal and stable clusters. The Wikipedia article, as usual, is a good place to start your research.

As an aside, one simple way to deal with the problem of varying results based on the random cluster initialization is to repeat the clustering algorithm many times and choose the "best" result.