# Cluster evaluation in system that implement auto determine cluster number

First of all, how to know wheter document clustering result was "good" or "bad"? Not in ordinary k-means, but in algorithm that enable automatic cluster number, like g-Means. Came across Q&A related to cluster evaluation, I found that I can implement F-measure to determine whether system giving "good" or "bad" result regarding to clustering. Case is : you want to cluster documents. If it's you that define number of cluster, say k = 5, then it's easy to find True Positive (TP), TN, FT, and FN. But what if my automatic system say there is k = 7, because you have some outlier documents that 'mathematically' don't match with any clusters, so the system create new cluster, thus adding number of clusters. Let's say system assign some outlier documents to cluster 6 and the rest to cluster 7. Since document relevance is very subjective to human, the human tester can say that outlier document belongs to cluster 5. If I use F-measure, This testing result could give my sistem very bad result (accuracy). Is there any other possible method? Thanks for your answer.

• can implement F-measure to determine whether... Are you implying that you have the "ground truth" information, that is, know the true clusters in your data? – ttnphns Jun 17 '16 at 10:39
• Accuracy and F-measure are quality metrics for classification, not clustering. What you may want to try is V-Measure – rcpinto Jun 17 '16 at 13:13