Well, I don't have labels (ground truth) for my data points. However, by domain knowledge, I am certain that a particular soft clustering algorithm will produce satisfactory results. Hence, I may use its output, denoted by C, as a surrogate of the ground truth. Let's say $$C=\{(0.1, 0.9), (0.2, 0.8, \ldots)\},$$ where $C_1=(0.1, 0.9)$ means that the 1st data point is in Cluster 1 with 0.1 probability and in Cluster 2 with 0.9 probability.

Now, I want to benchmark some other hard clustering algorithms against my "ground truth", $C$.

What is the rightful way of doing so? Is there an already-formed convention for this?


1 Answer 1


Most "external" (supervised) evaluation measures such as ARI should work, if you take element weights into account. Most are based on an agreement of pairs; and you may argue that a pair exists with $p_1\cdot p_2$ in your soft clustering, if $p_1$ and $p_2$ are the probabilities of objects 1 and 2.

They probably just become much more computationally intensive, as you can't just use the cluster intersection sizes anymore.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.