I estimate ratings in a user-item matrix by decomposing the matrix into two matrices P and Q and then using gradient descent to minimize the error. Now, if I want to add a new user, the most obvious solution is to retrain the model. This, however, takes a lot of time even with a small number of steps.

Singular Value Decomposition allows to easily add a new user by computing: user_k = Sigma^(-1) * Ut * user

Is there anything like that for matrix factorization? Can a new user be added without recomputing everything?

  • $\begingroup$ What do you mean 'add'? You want to incorporate them into the whole decomposition? $\endgroup$ – Jakub Bartczuk Jan 1 '18 at 21:45
  • $\begingroup$ @JakubBartczuk yes, I think so. I want to estimate ratings for a new user, and that seems to be the only way to do it. $\endgroup$ – lawful_neutral Jan 1 '18 at 22:13

Since your training matrix decomposition with gradient descent, I assume you have some loss function $L(X - PQ)$ where $L$ is squared Frobenius norm or something similar.

When you add a new user (let's say rows of $X$ correspond to users, and $x$ is new user's row, so that $X$' is $X$ with concatenated $x$) your objective becomes $$L(X' - P'Q)$$ and if your loss is something that can be calculated by summing over rows, $$L(X' - P'Q) = L(X - PQ) + L(x - x_pQ)$$.

If you already have trained $P, Q$ so that they minimize $L(X - PQ)$ then you have a guess for $P, Q$.

If you want to incorporate new user, but not recompute everything, you could either optimize $L(x - x_pQ)$ with $Q$ fixed or changed.

  • 1
    $\begingroup$ Thank you! So, basically, I just do the same optimization that I did for the whole matrix, but just for one row instead of all of them? I just implemented it with the fixed Q, it seems to be working fine. :) $\endgroup$ – lawful_neutral Jan 2 '18 at 1:16
  • $\begingroup$ I think this also applies to ALS-WR (at least I applied and seems to work) $\endgroup$ – Máthé Endre-Botond Jun 26 '20 at 2:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.