I have some trouble understanding the concept of using precision and recall to evaluate a content-based recommender.

Suppose I want to recommend articles to users. A content-based recommender will first get my article reading history, then find articles similar to what I have read based on some clustering result.

Here's a graph about precision and recall:

enter image description here

If I understand correctly, the true positives part is fixed, which is always the number of articles I've read. That means the the false negatives part is 0, thus recall is always 1?

  • $\begingroup$ It depends on your definition of true positive (i.e. a successful guess), hence your performance measure. this and this might be related. $\endgroup$ – jeff Jun 16 '16 at 22:18
  • $\begingroup$ @HalilPazarlama I guess a more appropriate definition for precision and recall for recommender systems would be, precision = (# of recommendations in the user's library)/(total number of recommendations), recall = (# of recommendations in the user's library)/(# of articles in the library). $\endgroup$ – Roy C Jun 16 '16 at 22:24
  • $\begingroup$ I see. then the number of false negatives should be the ones you returned but not in the user's library, which is not necessarily 0. right? $\endgroup$ – jeff Jun 16 '16 at 23:20

Sounds like you don't have labeled data. If I understand your problem correctly, a false negative would be an article that you predict would be "interesting", but that you have not yet read. If this is correct, then you would want to maximize your false negatives!

Read up on ways to evaluate unsupervised learning

Googled and found http://research.microsoft.com/pubs/115396/EvaluationMetrics.TR.pdf


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.