For supervised learning, both regression and classification have ground truth. The model performance can be measured against ground truth. For example, $R^2$ in regression or accuracy (0-1) in classification.

On the other hand, how can we measure the performance for clustering algorithm? (Let's assume number of clusters is given.)

There are two ways I can think of but want to validate with experts in the community and get more possible ways of measuring clustering performance.

  • The first way is using label. Labeled data can also be used in clustering, but when build the model, we do not use the label. Label will be used in evaluation. For example, we can clustering iris data based on flower features, but check the clustering results against label. A better model will clustering same type of flower in one cluster.

  • The second way is checking loss in test data set. Loss function is always a way to measure the model performance. For example, we use mixture of Gaussian to do the clustering and want to maximize the likelihood. We can split data into training and testing. A better algorithm will have a better loss in testing data.

Are these two ways reasonable and any other ways?

  • $\begingroup$ The answer can differ depending on the purpose of the performance measure (e.g. selecting the number of clusters, comparing different algorithms against each other, etc.). What type of purpose do you have in mind? $\endgroup$
    – user20160
    Jun 26 '16 at 1:45
  • 4
    $\begingroup$ In this thread with a few answers there are also links to similar questions already asked on the site. $\endgroup$
    – ttnphns
    Jun 26 '16 at 5:13