Updating SVD in Recommender Systems for change in ratings I have read that there are projection based methods to accomodate for new user's ratings or for the ratings for a new item in SVD.
However, I want to know how to update my feature space for change in ratings by existing users incrementally. That is, if user U has a feedback of 'r1' for item I, and if the feedback changes to 'r2', I don't want to re-run the entire SVD for this to get the new feature vectors immediately. I can stay with errors temporarily before I have too many new changes when I will re -run the model.
All the ratings are implicit behaviour feedbacks like number of buys/browses.
I have used Yehuda Koren's labs.yahoo.com/files/HuKorenVolinsky-ICDM08.pdf paper by using the Modified ALSE version of SVD which does not have an orthogonal space but the heart of it is the SVD based dimension reduction.
I want to know how to accomodate for the changes incrementally. Please help me.
 A: As someone who practically works with these systems, here is how I do it -
Let's say you have your fancy recommender system go ahead and decompose your matrix of users and ratings ($Y$) to users and factors ($X$) and products and factors ($\Theta$). So your prediction for these users is 
$Y = X * (\Theta)^T$.
Now you have some new users coming in (or updates to existing users). Let's call this $Y'$. Assuming your products have not changed by a lot, you can run gradient descent keeping $\Theta$ fixed and figuring out the new $X'$. Your cost function remains the same -
$C = .5 * [ (Y' - X' * \Theta^t)^2 + \lambda * |X'|^2 + \lambda * |\Theta|^2]$
But you only have to figure out the new $X'$ given the same $\Theta$. Your gradient descent update is now some factor of $ (Y' - X' * \Theta^t) * X + \lambda * \Theta$. Once you have the new $X'$ you can calculate the new $Y'$.
This works well in practice. You can update $\Theta$ or calculate the whole thing again as frequently as required (when you have enough new products, etc). This is basically breaking up the ALS steps.
