Evaluation of a recommendation system. How can I do that?

Question

I'm learning to make a book recommendation system but I am facing some difficulties to evaluate the model. I chose the collaborative filtering item based strategy. The dataset is a matrix (book, user) filled with the ratings of the books. The dataset is something like:

        user_a user_b user_c  ... user_x
book_1    0     3      5     ...  4
book_2    2     1      0     ...  0
book_3    0     0      0     ...  2
...

Bellow the code to train the model.

from sklearn.neighbors import NearestNeighbors
model = NearestNeighbors(metric='cosine', algorithm='brute')
model.fit(dataset)

# get the index of a book that contains 'harry potter' in its name
title = 'Harry Potter'
mask = books['title'].str.contains(title)
book_isbn = books[mask]['isbn']
mask = X.index.isin(book_isbn)
book_reference = X[mask].head(1)

# find the 5 nearest books from 'harry potter'
k = 5
distances, indices = model.kneighbors(book_reference.values, n_neighbors=k+1)

Well, the thing is 'kneighbors()' function is returning the distances of the 5 nearest vectors(books) from 'book_reference' and their indices. I don't know how evaluate the performance of this model since its not making predictions. How can I do this?

Answer 1

There are specialized metrics for recommender systems like Mean Percentage Ranking (MPR) and Mean Reciprocal Rank (MRR) and variations of the regular classification metrics like Precision@$k$ or Recall@$k$. You should look into those.