I have read that there are projection based methods to accomodate for new user's ratings or for the ratings for a new item in SVD. However, I want to know how to update my feature space for change in ratings by existing users incrementally. That is, if user U has a feedback of 'r1' for item I, and if the feedback changes to 'r2', I don't want to re-run the entire SVD for this to get the new feature vectors immediately. I can stay with errors temporarily before I have too many new changes when I will re -run the model. All the ratings are implicit behaviour feedbacks like number of buys/browses. I have used Yehuda Koren's labs.yahoo.com/files/HuKorenVolinsky-ICDM08.pdf paper by using the Modified ALSE version of SVD which does not have an orthogonal space but the heart of it is the SVD based dimension reduction. I want to know how to accomodate for the changes incrementally. Please help me.


As someone who practically works with these systems, here is how I do it -

Let's say you have your fancy recommender system go ahead and decompose your matrix of users and ratings ($Y$) to users and factors ($X$) and products and factors ($\Theta$). So your prediction for these users is

$Y = X * (\Theta)^T$.

Now you have some new users coming in (or updates to existing users). Let's call this $Y'$. Assuming your products have not changed by a lot, you can run gradient descent keeping $\Theta$ fixed and figuring out the new $X'$. Your cost function remains the same -

$C = .5 * [ (Y' - X' * \Theta^t)^2 + \lambda * |X'|^2 + \lambda * |\Theta|^2]$

But you only have to figure out the new $X'$ given the same $\Theta$. Your gradient descent update is now some factor of $ (Y' - X' * \Theta^t) * X + \lambda * \Theta$. Once you have the new $X'$ you can calculate the new $Y'$.

This works well in practice. You can update $\Theta$ or calculate the whole thing again as frequently as required (when you have enough new products, etc). This is basically breaking up the ALS steps.

  • $\begingroup$ Correct me if I am wrong, but my understanding of updating like this, is that it's most appropriate when the new rating is replacing the old one, and the so weights are learned that align the predictions towards the new rating over time. What happens in the case of implicit data (what the OP asked about), which would be cumulative? $\endgroup$ – Antimony May 4 '17 at 21:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.