2
$\begingroup$

I have read about SVD, and understand it as being similar to what PCA does. For recommendation, let's say I have a matrix where rows are users and columns are items, and the entry $(i,j)$ in matrix is rating given by user $i$ to item $j$, say 1-5 (discrete). Naturally, lots of entries are missing.

How to apply SVD in this case, since the matrix has missing data?

Imputing with either 0s or the global average seems to make no sense to me. So how do we proceed with SVD? Also, what are some modifications we could make to make SVD work better?

Improve this question
$\endgroup$
0
1
$\begingroup$

The answer is we don't really apply SVD in recommender systems. Let's check the classical paper (you should read it, it's a good read):

Matrix Factorization Techniques For Recommender Systems

Quote:

Such a model is closely related to singular value decomposition (SVD), a well-established technique for identifying latent semantic factors in information retrieval. Applying SVD in the collaborative filtering domain requires factoring the user-item rating matrix. This often raises difficulties due to the high portion of missing values caused by sparseness in the user-item ratings matrix. Conventional SVD is undefined when knowledge about the matrix is incomplete.

So SVD wouldn't be able to do it. It exists only in text-books. The authors suggest an alternative approach known as alternative least squares.

I have to be honest I read this paper a while ago, and don't remember exactly what ALS is. I'm not able to provide a description about ALS without reading the paper again. Maybe someone else can guide you if you need further help?

Improve this answer
$\endgroup$
7
  • $\begingroup$ Thanks for the input! Everywhere I read about SVD, it really gives no practical advice on how to implement it! I will go through this paper and see where that gets me. Though I was looking for some simple variation of SVD to handle this, given how frequently its mentioned in the literature. $\endgroup$ – user3676846 Apr 12 '17 at 8:12
  • $\begingroup$ @user3676846 SVD is useful only without missing data. $\endgroup$ – SmallChess Apr 12 '17 at 8:22
  • $\begingroup$ +1 for the link, but what do you mean "It exists only in text-books"? $\endgroup$ – amoeba Apr 12 '17 at 9:16
  • $\begingroup$ @amoeba In my answer I'd argue SVD wouldn't be very useful practically due to missing data. Text books talk about it because it builds foundation for learning more sophisticated models, but we can't really use it in the real world. Not without modification. $\endgroup$ – SmallChess Apr 12 '17 at 9:18
  • $\begingroup$ People talk about "SVD with missing data"; clearly it's not really classical SVD anymore, but people still call it SVD and I think it's fair enough. $\endgroup$ – amoeba Apr 12 '17 at 9:22

Not the answer you're looking for? Browse other questions tagged or ask your own question.