Why doesn't recall take into account true negatives? In experiments where true negatives are just as important as true positives, is their a comparable metric that does take it into account?
$\begingroup$
$\endgroup$
1
-
2$\begingroup$ I think True Negative Rate (also called specificity in other settings) is was you are looking for, but see en.wikipedia.org/wiki/Precision_and_recall. $\endgroup$– chlMar 9, 2011 at 18:45
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
Recall (in combination with precision) is generally used in areas where one is primarily interested in finding the Positives. An example for such an area is e.g. Performance Marketing or (as already suggested by ch'ls link) the area of Information Retrieval.
So:
If you are primarily interested in finding the negatives, "True Negative Rate" (as already suggested by chl) is the way to go. But don't forget to look at a "precision for focus on negatives"-metric (i.e. $\frac{TN}{TN + FN}$, because otherwise the "True Negative Rate" can be optimized by setting the prediction to "Negative" for all data points).
If you are interested in optimizing recall for both negatives AND positives, you should look at "Accuracy" (see again chl's link). But beware of class skew (i.e. you have many many more positives than negatives or vice versa ... in this case one can "optimize" accuracy by setting the prediction to the major class for all data points).
-
$\begingroup$ Thanks for the answer to my question. I am indeed interested in optimizing for both negatives and positives. In this case, it seems that accuracy is the way to go since it considers tp, fp, tn, and fn. However, as you mentioned above, I must be aware of the class skew. Thus, should I present accuracy along side another metric to counter this? Thanks again! $\endgroup$– khatchadMar 15, 2011 at 18:20
-
$\begingroup$ @Raffi: You could add the ratio of correctly classified examples of the minor class (i.e. precision or true negative rate respectively). However, I think it should be enough that you state your awareness of this problem and that you check that the model is not just predicting the major class. But this is just my opinion. $\endgroup$– steffenMar 16, 2011 at 7:18
-
$\begingroup$ Thanks! I think I will go that route then, i.e., the present just accuracy and that the model is not predicting just the major class. $\endgroup$– khatchadMar 17, 2011 at 15:59