I understand that f-measure (based on precision and recall) is an estimate of how accurate a classifier is. Also, f-measure is favored over accuracy when we have an unbalanced dataset. I have a simple question (which is more about using correct terminology than about technology). I have an unbalanced dataset and I use f-measure in my experiments. I am about to write a paper which is NOT for a machine learning/ data mining conference. Hence, can I refer to f-measure synonymously with accuracy in this context. For eg, I have a f-measure of 0.82, then can I say my classifier achieves 82% accurate predictions?

  • $\begingroup$ It would be better to introduce the f-measure if you use it. Substituting the two is not correct in my point of view. In your case if your accuracy is 99% you'll achieve 99% accurate predictions not matter what your f-measure is, and it could lead readers into error. $\endgroup$
    – AdrienNK
    Commented May 24, 2014 at 12:01
  • $\begingroup$ @AdrienNK: 99% accuracy does not imply 99% correct predictions unless the relative frequencies of the test cases are the same as in the real application situation. $\endgroup$ Commented May 24, 2014 at 14:41
  • $\begingroup$ @cbeleites you're right, I know, but often the test cases are issued from the same distribution (well maybe that's the biased view I have of it because I rarely had to work with data on which that wasn't the case) $\endgroup$
    – AdrienNK
    Commented May 24, 2014 at 15:01
  • $\begingroup$ @AdrienNK: I'm analytical chemist working towards medical diagnoses. The prevalence of the disease in question may vary about orders of magnitude between different patient subpopulations. See e.g. the discussion of the different PPVs in the second half of this article: nature.com/news/2011/110323/full/471428a.html $\endgroup$ Commented May 24, 2014 at 15:09
  • 2
    $\begingroup$ That was a fascinating read, thank you for bringing that to my attention. $\endgroup$
    – AdrienNK
    Commented May 24, 2014 at 17:15

2 Answers 2


First of all, I find "accuracy" sometimes a bit misleading, as it refers to distinct things:

The term accuracy in general for evaluating systems or methods (I'm analytical chemist) refers to the bias of predictions, i.e. it answers the question how good predictions are on average.

As you know, there are lots of different performance measures that answer different aspects of performance for classifiers. One of them happens to be called accuracy as well. If your paper is not for a machine learning/classification audience, I'recommend to make this distinction very clear. Even for this more specific meaning of accuracy I'd be very explicit of what I call accuracy as again several ways of dealing with class imbalance may occur. Typically, class imbalance is ignored, leading to the well-known $\frac{TP+TN}{all~cases}$ calculation. However, you may also use the average of sensitivity and specificity, which amounts to controlling class imbalance by weighting your average.

The F$_1$-score is often introduced as harmonic mean of precision and recall (or positive predictive value and sensitivity). For your question I think it helpful to spell this out a bit further and simplify it:

$F_1 = \frac{2 \cdot precision \cdot recall}{precision + recall} = \frac{2 \frac{TP}{all~P} \frac{TP}{all T}}{\frac{TP}{all~P} + \frac{TP}{all T}} = \frac{2 \frac{TP^2}{all~P \cdot all T}}{\frac{TP \cdot all~T}{all~P \cdot all T} + \frac{TP \cdot all~P}{all~P \cdot all T}} = \frac{2~TP^2}{TP \cdot all~T + TP \cdot all~P} = \frac{2~TP}{all~T + all~P} $

The last expression is not a fraction of anything that I can think of as a certain group of test cases. In particular, a (heavy) overlap between the TRUE and the POSITIVE cases is expected. This would keep me from expressing an F-score as percentage as that kind of implies a proportion of cases. Actually, I think I'd warn the reader that F-score does not have such an interpretation.

What makes me particularly uneasy combining these measure is that when we look at sensitivity = precision and specificity on the one hand, and positive and negative predictive values (PPV = recall) at the other hand, they are conditional probabilities, where sensitivity/precision and specificity are conditioned on the true class membership and the predictive values and recall are conditioned on the predicted class membership.

To get the latter from the former, we can use Bayes rule and the prior probabilities of the classes.

So F-score(s) average (by harmonic mean, but my point would stay the same with arithmetic or geometric mean) together figures of merit that are so-to-speak before and after the Bayes rule. I.e., we produce a metric that is neither completely here nor there.

  • 1
    $\begingroup$ more specifically this is $F_1$ measure. F-score can be generalized with a separate parameter $\endgroup$
    – qwr
    Commented Dec 9, 2018 at 7:19
  • $\begingroup$ @qwr: thanks, yes, updated. $\endgroup$ Commented Aug 16, 2022 at 12:19

Quick Answer:

No, F-measure formula does not consist of TN factor, and it is useful on retrieve problems (doc).

Thus, it's (F-measure) the correct approach to evaluate the imbalanced datasets or in retrieval problems case instead of accuracy and ROC.

Accuracy = (TP+TN) / (TP+FP+FN+TN)

F1_Score = 2*(Recall * Precision) / (Recall + Precision)
# or
F1_Score = 2*TP / (2*TP + FP + FN)


Precision = TP / (TP+FP)

Recall = TP / (TP+FN)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.