While I was reading Torgo's Data Mining with R, I found that the description of precision/recall curve was different compared with other approaches. Usually, these curves are based on a threshold that determines which probability value is good enough to decide when an event has occurred, so we can classify future events depending on that value. However, Torgo's description is as follows:
Precision/recall (PR) curves are visual representations of the performance of a model in terms of the precision and recall statistics. The curves are ob- tained by proper interpolation of the values of the statistics at different working points. These working points can be given by different cut-off limits on a ranking of the class of interest provided by the model. In our case this would correspond to different effort limits applied to the outlier ranking produced by the models. Iterating over different limits (i.e., inspect less or more reports), we get different values of precision and recall. PR curves allow this type of analysis.
The application the author has in mind is that of a fraud detection problem in which we have a classification task resulting in values
ok. We would like to output probabilities, rank them, select the first $k$ reports and be able to inspect them.
Is this an alternative measure of threshold in precision/recall curves? I think it is assuming that probabilities below 0.5 are to be classified as
ok, 0.5 is equivalent to
unknown and above 0.5 means
fraud. Is that a correct assumption to make?
Thanks a lot!