2
$\begingroup$

I'm not sure how to correctly calculate the standard deviation of recall, precision, accuracy and F-score.

For example:

TP  FP  FN  TN  total   recall   precision    F-Score    accuracy                           
40  10  15  50  115     0.727      0.800       0.762      0.783

TP = True Positive, FN = False Negative, etc.

(Do I use the formula for standard deviation of a binomial distribution: $\sigma = \sqrt{np(1-p)}$? OR $\sigma = \sqrt{\frac{p(1-p)}{n}}$ with $n$ = total and $p$ = recall, precision, F-Score, or accuracy.)

How can i calculate the standard deviation?

I'm not even sure if it makes sense but i was asked to check this.

$\endgroup$

1 Answer 1

0
$\begingroup$

In order to even think of calculating the SD from a single observation you have to know the distribution of the measure, and it is not the case for the majority of predictive models.

Thus, you are left with non-parametric ways of estimating SD; for instance, you can cross-validate the model and then use the vector of precision/recall/F/acc values over folds.

$\endgroup$
2
  • $\begingroup$ yes i think you're right about the cross validation. $\endgroup$
    – ddd
    Commented Feb 5, 2014 at 21:47
  • $\begingroup$ Hard to see this as a "single observation" when there are 115 cases observed. $\endgroup$
    – shabbychef
    Commented Nov 6, 2023 at 18:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.