I have a bunch of experiments in which I am calculating precision and recall. I want to present a mean precision and recall for these experiments. Should these values be weighted by anything?
It would be good to know more about the experiments. But there's two main ways (at least in information retrieval) of averaging contingency-table based measures like recall and precision:
Compute the individual measures for each experiment and take the unweighted average ("macroaveraging").
Add up the contingency tables and compute the measures from the summed contingency table ("microaveraging").
The usual reference on these is
Tague, J. The pragmatics of information retrieval experimentation. In Information Retrieval Experiment, Butterworths, London, 1981, pp 59-102.
Yang, Rose, Li, and I also discuss these two approaches, and provide sample data sets in RCV1: A New Benchmark Collection for Text Categorization Research.