I'm learning to make a book recommendation system but I am facing some difficulties to evaluate the model. I chose the collaborative filtering item based strategy. The dataset is a matrix (book, user) filled with the ratings of the books. The dataset is something like:

        user_a user_b user_c  ... user_x
book_1    0     3      5     ...  4
book_2    2     1      0     ...  0
book_3    0     0      0     ...  2

Bellow the code to train the model.

from sklearn.neighbors import NearestNeighbors
model = NearestNeighbors(metric='cosine', algorithm='brute')

# get the index of a book that contains 'harry potter' in its name
title = 'Harry Potter'
mask = books['title'].str.contains(title)
book_isbn = books[mask]['isbn']
mask = X.index.isin(book_isbn)
book_reference = X[mask].head(1)

# find the 5 nearest books from 'harry potter'
k = 5
distances, indices = model.kneighbors(book_reference.values, n_neighbors=k+1)

Well, the thing is 'kneighbors()' function is returning the distances of the 5 nearest vectors(books) from 'book_reference' and their indices. I don't know how evaluate the performance of this model since its not making predictions. How can I do this?


1 Answer 1


There are specialized metrics for recommender systems like Mean Percentage Ranking (MPR) and Mean Reciprocal Rank (MRR) and variations of the regular classification metrics like Precision@$k$ or Recall@$k$. You should look into those.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.