Summary statistics of the precision-recall curve From what I understand, one can use the AUC of the ROC curve as a summary statistic of the full curve. 
Q1. Are there any similar summary statistics that one can use on a single precision-recall curve?
Q2. As far as I understand, the $F$-score ($F_1$ or $F_\beta$), is measured at a specific operating precision-recall regime. i.e. one needs to fix a point on the precision-recall curve to obtain a precision rate and it's associated recall rate to obtain a result in the formulas:
$F_1 = 2 \cdot \frac{\mathrm{precision} \cdot \mathrm{recall}}{\mathrm{precision} + \mathrm{recall}}$
$F_\beta = (1 + \beta^2) \cdot \frac{\mathrm{precision} \cdot \mathrm{recall}}{(\beta^2 \cdot \mathrm{precision}) + \mathrm{recall}}$
If so, how does one choose such point on the curve?
 A: The "Mean Average Precision" (sometimes abbreviated mAP or MAP) might be what you want. It's pretty commonly used for evaluating information retrieval systems and is fairly straightforward to compute.
First, calculate the average precision for a given query. To do this, rank the documents and compute the precision after retrieving each relevant document. For example, suppose that four documents are relevant to this query, and our system returned the following:


*

*Relevant document

*Irrelevant document

*Relevant document

*Relevant document

*Irrelevant document

*Irrelevant document. 

*Relevant document


The first relevant document is at position one, and the precision there is 1/1 = 1.0
The next relevant document is at position 3; two of the three documents seen so far are relevant, so our precision here is 2/3. Document 4 is relevant too and the precision score here is 3/4. The final relevant item is at position seven, giving us a precision of 4/7. 
Find the mean of  these precision scores (1/4*(1 + 2/3 + 3/4 + 4/7) = ~0.747) to get the average precision for this query. The mean average precision is just the mean of these averages across all the queries in your evaluation set.
As for choosing a precision-recall trade off, that's largely up to you. The $F_1$ score gives them equal weight; you can interpret the $\beta$ in $F_\beta$ as giving $\beta$ times more weight to recall than precision. I believe that some studies indicate that users prefer precision to recall, but I would bet that it depends a lot on the application and use-case. I certainly don't need google to show me every webpage about cats, but do want all the sites on the first page to be relevant. On the flip side, it might be more important to return every possibly-relevant document if you're doing discovery for a court case.
A: Actually there is just an AUC of PR-curve measure; it is used in biology (especially in the DREAM challenge series surroundings) because it is consistent with AUROC (i.e. the ranking is usually the same if the performance differs significantly) still gives better numerical resolution by giving lower values than AUROC. 
The problem is that AUPR requires careful integration, so it is pretty hard to find a correct implementation.
This is a canonical paper about the topic.
A: You can calculate the AUC of the ROC for just a single (precision,recall) data point.
This paper, Robust classification for imprecise environments, describes how to calculate the convex hull AUC (which is pretty standard now). When you have only one (precision,recall) point, you extend a straight line down to the always-say-no (0,0) point and a straight line up to the always-say-yes (1,1) point, and you have the convex hull.
Now the neat result: in this case, with only one coordinate, the calculation simplifies to
$AUC = (t − f + 1)/2 $.
This emphasises the connection between AUC and the Gini coefficient, remarked on elsewhere.
