I'm using the 100k MovieLense dataset to build a recommendation system in R, using the
From what I understood, the
recommenderlab package kind of forces the user to use the following evaluation technique.
- Split the dataset into training and test sets.
- Split the test set into known and unknown, base in the
This is what's done when I run the following:
library(recommenderlab) data(MovieLense) MovieLense <- binarize(MovieLense, minRating=4) MovieLense <- MovieLense[rowSums(MovieLense) > 4,] eva <- evaluationScheme( data = MovieLense, method = "split", train = 0.75, given = 3 )
So I my folowing work is:
- Train the recommender using the traing set
- Predict rating for users in the test set using the known data
- Evaluate prediction accuracy in the unknown test set data
This is performed using:
rec <- Recommender(getData(eva, "train"), method = "IBCF") pred <- predict(rec, getData(eva, "known"), type = "topN", n = 5) error <- calcPredictionAccuracy(pred, getData(eva, "unknown"), given = 3)
After all this, my question is:
- Is this a good evaluation technique?
I'm not confortable in using always only 3 given items to predict users ratings, because I think the algorithm could have a better performance if I used more ratings in prediction.
But, I know that if I use more "given" items, my precision will be smaller because there will be less relevant items too.
So, how do I choose the best number of items?
If this is a wrong way the evaluate my system:
- What's the best way to measure my algorithm performance?