I used K-means to cluster a large data set that has millions of samples. I tried to create the clusters with different sets of attributes, which, as a result, generated different optimal number of clusters. For example, using attributes A,B,C,D, 5 clusters were created while using attributes X,Y,Z, 4 clusters were created.

My questions are:

  1. How to compare and choose between these two clustering results considering they have different number of clusters and were created with different attributes?
  2. Is there a good metric to use?
  3. Any suggestion for R package that works well for the large data set?
  • 1
    $\begingroup$ Regarding specifically internal clustering validation indices - I would suggest you to read introductory paragraphs (about when we may use them) in my document. Find "Clustering criterions" on my web-page. $\endgroup$ – ttnphns Jun 14 '18 at 18:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.