Average precision/recall of multiple classifiers on the same dataset?

If I have some N human classifiers that can only predict in terms of 0 or 1 (not probabilistic, and also disregarding their own uncertainty. They either know or they don't), and each yield different precision/recall metrics for the same dataset, is there anything wrong with saying the average precision is then simply the arithmetic mean?

$$\frac{0.95 + 0.82 + 0.92}{3} = 0.90$$

Or is it "better" to calculate a final score from the majority vote (and why? Doesn't this add extra steps to calculate model uncertainty)?

• You mean, other than the "general" problems with accuracy (and precision)? Why is accuracy not the best measure for assessing classification models? More to the point, what is your goal in averaging accuracies? If you have multiple classifiers, you could let them vote (or - much better - combine their probabilistic predictions) and typically get better results. Sep 25, 2019 at 13:03
• Good suggestions. Rephrased problem. Sep 25, 2019 at 13:09
• Hm. Thanks for the edit. I was going to answer that there is nothing wrong with the "average precision" - until I saw your last paragraph. What is "besser" depends on what you want to do. Averaging precisions is not the same as counting votes to classify (and possibly calculating the precision of this ensemble classifier). You can average precisions in the way you propose. But what is your purpose in doing so? Is this just for information? Sep 25, 2019 at 13:13
• The purpose is basically to compare humans to a trained classification model, and also get an estimate of the variability between different humans doing the classification. Sep 25, 2019 at 13:28