12
$\begingroup$

Consider collaborative filtering problem. We have matrix $M$ of size #users * #items. $M_{i,j} = 1$ if user i likes item j, $M_{i,j} = 0$ if user i dislikes item j and $M_{i,j}=?$ if there is no data about (i,j) pair. We want to predict $M_{i,j}$ for future user, item pairs.

Standard collaborative filtering approach is to represent M as product of 2 matrices $U \times V$ such that $||M - U \times V||_2$ is minimal (e.g. minimizing mean square error for known elements of $M$).

To me logistic loss function seems more suitable, why are all algorithms using MSE?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ In this case it makes sense but most of the time M_i,j can be a rating and in that case the MSE is more useful. I'd say that the MSE is more general. $\endgroup$
    – ThiS
    Commented Sep 6, 2013 at 12:56

2 Answers 2

Reset to default
12
$\begingroup$

We use logistic loss for implicit matrix factorization at Spotify in the context of music recommendations (using play counts). We've just published a paper on our method in an upcoming NIPS 2014 workshop. The paper is titled Logistic Matrix Factorization for Implicit Feedback Data and can be found here http://stanford.edu/~rezab/nips2014workshop/submits/logmat.pdf

Code for the paper can be found on my Github https://github.com/MrChrisJohnson/logistic-mf

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ L(R | X, Y, β) = Prod( p(lui | xu, yi, βu, βi)^α.r_ui * (1 − p(lui | xu, yi , βu, βi))^ (1 - α.r_ui) A had a look to your code, and you do use 1+α.r_ui l64 : A = (self.counts + self.ones) * A github.com/MrChrisJohnson/logistic-mf/blob/master/… Therefore, am i missing something? Kind regards $\endgroup$
    – fstrub
    Commented Apr 20, 2015 at 10:33
  • $\begingroup$ I had a look to the paper you published. It is very interesting since matrix factorization with logistic regression has not been not actively studied. Anyway, I am a bit confused with your Loss function (2) L(R | X, Y, β) = Prod( p(lui | xu, yi, βu, βi)^α.r_ui * (1 − p(lui | xu, yi , βu, βi)) Regarding (3), I think that there is a typo mistaske L(R | X, Y, β) = Prod( p(lui | xu, yi, βu, βi)^α.r_ui * (1 − p(lui | xu, yi , βu, βi))^ (1 + α.r_ui) But, actually, I am still a bit confused. Indeed, I would have expected a Bernouilli-like law such as $\endgroup$
    – fstrub
    Commented Apr 20, 2015 at 10:33
  • $\begingroup$ Maybe I'm quite late on the subject.. someone had the chance to try this algo outside the context of music recommendation and instead the classical context of product recommendation? Thanks. $\endgroup$
    – Marco Fumagalli
    Commented Dec 19, 2018 at 16:38
5
$\begingroup$

Most of the papers you'll find on the subject will deal with matrices where the ratings are on a scale [0,5]. In the context of the Netflix Prize for example, matrices have discrete ratings from 1 to 5 (+ the missing values). That's why the squared error is the most spread cost function. Some other error measures such as the Kullback-Leibler divergence can be seen.

Another problem that can occur with standard matrix factorization is that some of the elements of the matrices U and V may be negative (particularly during the first steps). That's a reason why you wouldn't use the log-loss here as your cost function.

However, if you're talking about Non-negative Matrix Factorization you should be able to use the log-loss as your cost function. You are in a similar case than Logistic Regression where log-loss is used as the cost function: your observed values are 0's and 1's and you predict a number (probability) between 0 and 1.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.