2
$\begingroup$

I'm using the 100k MovieLense dataset to build a recommendation system in R, using the recommenderlabpackage.

From what I understood, the recommenderlab package kind of forces the user to use the following evaluation technique.

  • Split the dataset into training and test sets.
  • Split the test set into known and unknown, base in the given argument

This is what's done when I run the following:

library(recommenderlab)
data(MovieLense)
MovieLense <- binarize(MovieLense, minRating=4)
MovieLense <- MovieLense[rowSums(MovieLense) > 4,]

eva <- evaluationScheme(
  data = MovieLense,
  method = "split",
  train = 0.75, 
  given = 3 
)

So I my folowing work is:

  • Train the recommender using the traing set
  • Predict rating for users in the test set using the known data
  • Evaluate prediction accuracy in the unknown test set data

This is performed using:

rec <- Recommender(getData(eva, "train"), method = "IBCF")
pred <- predict(rec, getData(eva, "known"), type = "topN", n = 5)
error <- calcPredictionAccuracy(pred, getData(eva, "unknown"), given = 3)

After all this, my question is:

  • Is this a good evaluation technique?

I'm not confortable in using always only 3 given items to predict users ratings, because I think the algorithm could have a better performance if I used more ratings in prediction.

But, I know that if I use more "given" items, my precision will be smaller because there will be less relevant items too.

So, how do I choose the best number of items?

If this is a wrong way the evaluate my system:

  • What's the best way to measure my algorithm performance?
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.