I have read about SVD, and understand it as being similar to what PCA does. For recommendation, let's say I have a matrix where rows are users and columns are items, and the entry $(i,j)$ in matrix is rating given by user $i$ to item $j$, say 1-5 (discrete). Naturally, lots of entries are missing.
How to apply SVD in this case, since the matrix has missing data?
Imputing with either 0s or the global average seems to make no sense to me. So how do we proceed with SVD? Also, what are some modifications we could make to make SVD work better?