6
$\begingroup$

F-beta score's formula calculates like this:

$$ F_{\beta} = (1+ \beta^2) \frac{PR}{\beta^2P + R} $$

However, according to some sources, in case I want to add more emphasis to Precision I should use beta < 1, and complementary, in case I want to add less emphasis to Precision than Recall, I should use beta > 1. This somehow seems me contrary to the purpose; why to downweight beta in the denominator when I actually want to assign more weight to Precision in the caluculation?

*Bonus question: either way the answer is for the question above, is there any particular formula to define beta if I want to assign different weights to Precision and Recall? Maybe in proportion to the cost difference in false negatives and false positives? Or simply just use beta=0.5 and beta=2 as a rule of thumb?

$\endgroup$
1
  • $\begingroup$ I made some changes in the $F_\beta$ formula mentioned as it was misleading (missing parenthesis as well as having an addition when it should have been a multiplication). $\endgroup$
    – usεr11852
    Commented Aug 15, 2020 at 7:49

1 Answer 1

2
$\begingroup$

By setting $\beta$ to $0$ we are effectively getting "just Precision" that is because the Recall multipliers cancel each other out and we are left with Precision only. By using a lower $\beta$ we do not down-weight Precision in any way, we are up-weighting as we allow it to dominate the fraction's final value through the numerator.

If we already know the cost of our FP/FP we can directly use them. $\beta$ itself reflects our trade-off between Recall and Precision in the sense of $\beta=\frac{\text{Recall}}{\text{Precision}}$; therefore the values of $0.5$ and $2$ merely reflect that we hypotheises that: "we value Precision twice as much as Recall" (for $\beta=0.5$) or "we value Recall twice as much as Precision" (for $\beta=2$). Obviously if we value them equally $\beta=1$ and we get our standard $F_1$ score.

Sasaki (2007) The truth of the F-measure presents this discussion very nicely in a formal manner where it grounds it firmly on van Rijsbergen's original 1979 work on Information Retrieval.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.