Precision and recall in content-based recommender

Question

I have some trouble understanding the concept of using precision and recall to evaluate a content-based recommender.

Suppose I want to recommend articles to users. A content-based recommender will first get my article reading history, then find articles similar to what I have read based on some clustering result.

Here's a graph about precision and recall:

If I understand correctly, the true positives part is fixed, which is always the number of articles I've read. That means the the false negatives part is 0, thus recall is always 1?

Answer 1

Sounds like you don't have labeled data. If I understand your problem correctly, a false negative would be an article that you predict would be "interesting", but that you have not yet read. If this is correct, then you would want to maximize your false negatives!

Read up on ways to evaluate unsupervised learning

Googled and found http://research.microsoft.com/pubs/115396/EvaluationMetrics.TR.pdf

Answer 2

Paying attention to the meaning of the words "precision" and "recall" makes it considerably easier to understand them. Recall simply means how many of the points that are positive you recall correctly, i.e. the proportion of positives that you actually lable as positive.

Precision is simply the precision of your predictions, i.e. if you predict $n$ times that something has a positive lable, the precision is the proportion of making a correct decision when labelling something as positive

Answer 3

The usual scenario for a recommender system would be that for a particular user it scores the items in terms of how likely are they to be preferred by the user (clicked, watched, listened, purchased, etc). You do this by training your algorithm on the items the user interacted with (watching history on Netflix) to score the items that the user didn't interact with. To test the model, you would usually hold out some items per user (pretend to your model that the user didn't watch this movie) and ask the model for predictions. The recommender system metrics would judge if your model gives high scores to the items user actually has preferred and low scores for the items they didn't. To to this, there are specialized metrics, but you can also use metrics such as Recall@$k$: recall calculated when marking the $k$ items with highest scores as "positive" prediction.