4
$\begingroup$

I would like to know if there are any evaluation frameworks of recommender systems which are capable of evaluating rating prediction and topN recommendation (Precision and recall etc.). Maybe I need to find them in recommender frameworks? If so are they easy to plug in?

As a matter of fact I am working on dynamic recommender systems which may need the items for training are earlier than item for testing/recommendations, which is also the real application scenario I think.

In my point of view, as long as the data records have timestamps, the evaluation work should use training and testing items with time order. But many of the two sets of items are randomly selected. Am I right?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

Yes, try the R package recommenderlab.

shameless self-promotion - http://sanealytics.wordpress.com/2012/06/10/testing-recommender-systems-in-r/

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks, it is very useful information. I read the paper recommenderlab: A Framework for Developing and Testing Recommendation Algorithms. But for my second requirement about evaluation, it seems in the evaluation part of this toolkit, when choosing items to be withheld in the test data, the selection is random (section 3). Is it easy to change codes of this framework to make sure the selection of items follows my own protocol? I don't know R yet.. $\endgroup$
    – CyberPlayerOne
    Commented Jul 23, 2013 at 6:27
  • $\begingroup$ I haven't tried that yet but it can be done. There are two ways to do this - evaluationScheme creates an object of class evaluationScheme. The code for how it does that is r-forge.r-project.org/scm/viewvc.php/pkg/recommenderlab/R/… You can overwrite the default method, create your own or simply overwrite the slots of the object (not recommended as it is unsafe but easy to do). The other way is to divide into test and train matrices ahead of time and do your own evaluation. Another tool is VowpalW (See Matrix Factorization) $\endgroup$
    – ignorant
    Commented Jul 23, 2013 at 17:28
  • $\begingroup$ Thank you very much. Your answer is very helpful. I started learning R now :) $\endgroup$
    – CyberPlayerOne
    Commented Jul 30, 2013 at 1:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.