0
$\begingroup$

is it possible for an algorithm A to have a better precision but worse recall (or better recall but worse precision) than another algorithm B?

Although I know that precision and recall are different things, it seems to me that if algorithm A has a better precision (or recall) than algorithm B then algorithm A will also have a better recall (or precision) than B.

Thanks Ahmet

$\endgroup$
2
$\begingroup$

Typically, there is a tradeoffs between precision and recall, so yes.

Here's a simple example. Imagine you have predicted probabilities from a logistic regression, and you are choosing a classification threshold. A higher threshold will typically have better precision and worse recall than a low threshold.

$\endgroup$
2
  • $\begingroup$ I know that there is a trade-off between precision and recall. My question is not about the relation between precision and recall for a single algorithm but for two algorithms. But I think the current formulation of my problem is not clear, I will try to correct it $\endgroup$ Mar 3 '13 at 16:35
  • 1
    $\begingroup$ @Ahmet Yılmaz: The point I am making is: algorithm A may "prioritize" precision, while algorithm B may prioritize recall. It is not at all surprising to get a result like yours. Try comparing the 2 algorithm on a non-threshold dependent metric, like area under the ROC curve. $\endgroup$
    – Zach
    Mar 3 '13 at 19:50
3
$\begingroup$

To make an extreme example...

Algorithm A: always say yes (ie, label all examples as positive).

Algorithm B: only say yes in the one instance you are absolutely sure of.

Algorithm A has perfect recall (but usually pretty bad precision) and algorithm B has perfect precision--assuming that one instance was right--(but awful recall).

$\endgroup$
2
  • $\begingroup$ I agree with you, thank you for your answers. However, the point that I was try to make is somewhat different. But I think it will take some thinking to reformulate my question. Thanks any way. $\endgroup$ Mar 3 '13 at 20:02
  • 2
    $\begingroup$ @AhmetYılmaz Nothing says "Thanks" like an upvote. $\endgroup$ Mar 3 '13 at 20:06
2
$\begingroup$

Yes

In information retrieval, precision and recall are used to evaluate search algorithms. For example, if I were to have a fixed test database of 1000 books. Of those 1000 only 10 are from the the author Steven King. A high precision search algorithm would only return books written by Steven King but it would probably not return all of them, maybe seven of them. A high recall algorithm would return all 10 of the Steven King books but it would also return books by other authors that have name King or Steven. Balancing both precision and recall is a key concept in Information retrieval. Typically the weighted harmonic mean of precision over recall is used. It is called the F-Measure.

$\endgroup$
1
  • $\begingroup$ In a way, isn't that similar to false-positives and false-negatives? Or specificity and sensitivity? Is the difference only semantics? $\endgroup$ Mar 5 '13 at 7:10
1
$\begingroup$

Refer to this sample Precision-Recall curve plotted for evaluating the classification performance of 3 different algorithms on the same test set. Precision is plotted on the Y-axis and Recall on the X-axis, percentages have been expressed in decimals.

Precision recall curve

The curve is plotted by steadily increasing the cutoff value from 0 on the right-most side (recall=100%) to the highest value on the left-most side of the graph (recall=0%). For simplicity, we'll refer to the algorithms as "red", "green" and "blue".

Now, when we compare points A and B, the recall reduced by 20% when the cutoff was increased from point A to point B, while the precision increased by 4% since false-positive rate was reduced. But the corresponding precision for algorithm "green" was much worse indicating a poorer classification performance. When we examine the performance at point C, the "red" algorithm performs much better (higher precision) than the other two for the same level of recall.

Based on the nature of the classification task and the "cost" of false positives vs. false negatives, an appropriate algorithm may be chosen. For example, if a recall of 30% is the minimum acceptable for the task at hand, then algorithm "red" should be chosen since it has the highest precision, i.e. it will pick up more true positives for the same number of predicted positives.

Lets ignore algorithm "red" for a moment and consider only the "blue" and "green" classifiers. If the minimum acceptable recall is 70%, then algorithm "blue" would be chosen since it is more precise at that rate of recall. But if the minimum acceptable recall is 20%, then algorithm "green" should be chosen since it has higher precision at that recall rate.

It is possible for an algorithm to have better precision and recall than another algorithm and this can be indicated by plotting their curves together. For the same algorithm, usually if cutoff is raised precision increases while recall decreases. However, a lot depends on the algorithm being used and how well is it able to classify the dataset based on the attributes provided.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.