10
$\begingroup$

Why doesn't recall take into account true negatives? In experiments where true negatives are just as important as true positives, is their a comparable metric that does take it into account?

$\endgroup$
7
$\begingroup$

Recall (in combination with precision) is generally used in areas where one is primarily interested in finding the Positives. An example for such an area is e.g. Performance Marketing or (as already suggested by ch'ls link) the area of Information Retrieval.

So:

If you are primarily interested in finding the negatives, "True Negative Rate" (as already suggested by chl) is the way to go. But don't forget to look at a "precision for focus on negatives"-metric (i.e. $\frac{TN}{TN + FN}$, because otherwise the "True Negative Rate" can be optimized by setting the prediction to "Negative" for all data points).

If you are interested in optimizing recall for both negatives AND positives, you should look at "Accuracy" (see again chl's link). But beware of class skew (i.e. you have many many more positives than negatives or vice versa ... in this case one can "optimize" accuracy by setting the prediction to the major class for all data points).

$\endgroup$
  • $\begingroup$ Thanks for the answer to my question. I am indeed interested in optimizing for both negatives and positives. In this case, it seems that accuracy is the way to go since it considers tp, fp, tn, and fn. However, as you mentioned above, I must be aware of the class skew. Thus, should I present accuracy along side another metric to counter this? Thanks again! $\endgroup$ – Raffi Khatchadourian Mar 15 '11 at 18:20
  • $\begingroup$ @Raffi: You could add the ratio of correctly classified examples of the minor class (i.e. precision or true negative rate respectively). However, I think it should be enough that you state your awareness of this problem and that you check that the model is not just predicting the major class. But this is just my opinion. $\endgroup$ – steffen Mar 16 '11 at 7:18
  • $\begingroup$ Thanks! I think I will go that route then, i.e., the present just accuracy and that the model is not predicting just the major class. $\endgroup$ – Raffi Khatchadourian Mar 17 '11 at 15:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.