I'm reading about matrix factorization for recommender systems. A basic matrix factorization model would be something like: $(p_i \times q_j ) + b_i + b_j$. That formula would compute the rating for one element in the rating matrix. Now I can see that this is very similar to a usual logistic regression formula where you multiply the feature vector with a weight vector and add a bias terms. That would give you the formula of a hyperplane. But with the matrix factorization formula you have another bias terms. So can you explain to me what it does?
$\begingroup$
$\endgroup$
0
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
Basically in a factorization model you usually incorporate both bias and interaction terms.
Bias terms describe the effect of one dimension on the output. For example, in the Netflix challenge example, the bias of a movie would describe how well this movie is rated compared to the average, across all movies. This depends only on the movie (as a first approximation) and does not take into account the interaction between an user and the movie.
Similarly an user's bias corresponds to that user's tendency to give better or worse ratings than the average.
Finally the interaction term ($p_i \times q_j$ in your notation) describes the interaction between the user and the movie, ie. the user's preference for that movie.
The point of integrating bias terms is to remove the "bias" given by users (or the bias for a movie) : for example if Alice tends to rate all movies 1 star too high, in order to compare her ratings to other users, you must remove 1 star to all of her ratings. Similarly, if you want to predict the rating Alice will give to a movie, you have to add 1 star to the score you would get if Alice was an "average" user.
$\begingroup$
$\endgroup$
it might help to compare to regular linear regression with interaction terms (and ignore the stupid bias terminology).
you could build a linear regression model (with categorical variables for person and movie)
rating=a(i_person) + b(j_movie) + c(i_person,j_movie)
the $b_i$ and $b_j$ are your linear regression coefficients for person 'factor', and movie 'factor'. The problem comes with estimating c: each person has only seen a handful of films. so matrix factorization is saying lets replace the interaction term $c(i,j)$ with $p_i\times q_j$, and so regularising the problem.
$\begingroup$
$\endgroup$
Let's say you have a user who hasn't rated any movies.
Assuming your $p_i$ is the factor for the user, it would come down to 0. This means that your $p_i * q_i$ will predict no movies for this poor user. You atleast want to give her the average rating, don't you?
That's basically what the bias term does. It takes out the average movie bias and the user bias (some people might be harsher raters than others). So you basically figure them out as well as you figure out how much the user likes that movie without those factors.
In practice, this means that you might end up recommending Titanic slightly more than you would like to (as its bias is high), but it's a safe choice whether you like it or not.
