1
$\begingroup$

First of all, how to know wheter document clustering result was "good" or "bad"? Not in ordinary k-means, but in algorithm that enable automatic cluster number, like g-Means. Came across Q&A related to cluster evaluation, I found that I can implement F-measure to determine whether system giving "good" or "bad" result regarding to clustering. Case is : you want to cluster documents. If it's you that define number of cluster, say k = 5, then it's easy to find True Positive (TP), TN, FT, and FN. But what if my automatic system say there is k = 7, because you have some outlier documents that 'mathematically' don't match with any clusters, so the system create new cluster, thus adding number of clusters. Let's say system assign some outlier documents to cluster 6 and the rest to cluster 7. Since document relevance is very subjective to human, the human tester can say that outlier document belongs to cluster 5. If I use F-measure, This testing result could give my sistem very bad result (accuracy). Is there any other possible method? Thanks for your answer.

$\endgroup$
  • $\begingroup$ can implement F-measure to determine whether... Are you implying that you have the "ground truth" information, that is, know the true clusters in your data? $\endgroup$ – ttnphns Jun 17 '16 at 10:39
  • $\begingroup$ Accuracy and F-measure are quality metrics for classification, not clustering. What you may want to try is V-Measure $\endgroup$ – rcpinto Jun 17 '16 at 13:13
0
$\begingroup$

You can only compute the F measure if you have labeled data. But then you should be using classification, not clustering.

When using clustering, any use of labels is questionable. Because you say "the best algorithm produces what I already know", but then you don't need to run it at all! Whereas a clustering tjat e.g. found clusters within your labels (probably a much better clustering result) will perform bad on your measure!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.