1
$\begingroup$

I'm developing the Java application for text document clustering, and I'm researching some evaluation methods. I implemented F-measure (http://en.wikipedia.org/wiki/F1_score), but I have a problem - the returned value is NaN. It happens where a cluster doesn't contain any data from a specific category - precision and recall are equal to zero. How should I handle this situation - F-measure in that case should be zero as well? I will be very grateful for any advice.

$\endgroup$

2 Answers 2

1
$\begingroup$

Short answer: I would just have an if statement that checks if both the precision and recall are zero and set the F-score to zero when that occurs.

Long answer: In a rigorous mathematical sense, the F1-score is defined such that if the precision and recall are both zero, the F1-score is undefined: $$ F1=2\cdot\frac{precision \cdot recall}{precision+recall}. $$

You could perhaps justify setting F1 to zero by arguing that you are only sampling the true population and as precision and recall approach a small value, $\epsilon$, the numerator approaches zero faster than the denominator since it goes like $\frac{\epsilon^{2}}{\epsilon}=\epsilon$~0.

$\endgroup$
0
$\begingroup$

There is more than one F-measure around in the sense that it is computed on different data.

For evaluating cluster analysis, it seems to be most common to compute the F-measure on pairs of objects, and on the complete data set, not on single clusters.

See for example: https://stackoverflow.com/questions/12725263/computing-f-measure-for-clustering

$\endgroup$
2
  • $\begingroup$ I know, I'm calculating a weighted mean of F-measures for all clusters later. $\endgroup$ Jun 9, 2013 at 12:41
  • $\begingroup$ Precision and recall for individual clusters aren't clearly defined or sensible, that is the problem you are running into. $\endgroup$ Jun 9, 2013 at 15:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.