Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
We can compute the F-1 score in the following two ways.
$F_{1_{PRE, REC}} = 2 * (PRE * REC) / (PRE + REC)$
$F_{1_{TP, FP, FN}} = (2 * TP) / (2 * TP + FP + FN)$
Both computes F1 score, but which one is more appropriate to use in practice to classify highly imbalanced data with binary response? I believe Sklearn python library uses the first one.Thanks!
The formulas are exactly equivalent, as you can see from the Wikipedia page. So if you want to calculate an F1 score, it doesn't matter which you choose as you get the same result.
As to being "appropriate," many who frequent this site would argue that the F1 score isn't "appropriate" in most circumstances. It is based on an arbitrary choice of a probability cutoff for making the class assignments (typically a hidden assumption of p = 0.5 for the cutoff), and it implicitly assumes that recall and precision are equally important. To deal with the latter issue, you at least should calculate the $F_{\beta}$ score when "recall is considered $\beta$ times as important as precision" (Wikipedia):
$$F_\beta = (1 + \beta^2) \cdot \frac{\mathrm{precision} \cdot \mathrm{recall}}{(\beta^2 \cdot \mathrm{precision}) + \mathrm{recall}} $$
Furthermore, all the F scores ignore the true negatives. Thus if you switch the choice of the positive class you get a different F score. Implications of that problem and suggestions for better model-assessment approaches are on this page and on many others linked from a comment on the present question. Build a good probability model based on a proper scoring rule, and only then choose a probability cutoff based on the relative costs of false-positive and false-negative decision. If you then want to report an F score, fine, but don't use it for building and assessing the model.
