I have a sparse matrix. There is lots of missing data. Hence, I can't use SVD naively.
I read Koren's SVD++ paper. I'm confused as to how the $q_i$ and $p_u$ vectors are determined. $q_i^Tp_u$ is supposed to capture the interactions between user $u$ and item $i$ (plus some biases). I just don't see how to calculate what $q_i$ and $p_u$ are supposed to be. The most natural things would be to use SVD but you can't since it has missing data.
The SVD theory is only used for motivation. In reality, SVD is not defined for a sparse matrix. For netflix there is less than 1% fille data so this tends to be an issue.
You are right that qi and pu are user and item matrices. He then uses gradient descent to solve them as a cost function.
$q_i$ is a vector of $f$ dimensions, each value $q_{ifj}$ is the score on dimension $f_j$ of item $i$
likewise, $p_u$ is a vector of $f$ dimensions, each value $p_{ufj}$ is the extent to which the user $u$ is interested in dimension $f_j$
The physical meaning of inner product is projection of vector $i$ to vector $j$
so, $r = \mu + b_i + b_u + q_i^T * p_u$
and we could get $q_i$ and $p_u$ by using stochastic gradient descent or alternative least square method.
In this problem matrix factorization model maps both users and items to a joint latent factor space of f dimensions. $q_i$ and $p_u$ are the latent factors in this context. Here problem is to estimate the values of $q_i$ and $p_u$ so that the the interaction of $q_i$ and $p_u$ i.e. $q_i^Tp_u$ can approximate the same rating matrix for the known data instances. Remaining missing values will automatically be imputed with the model.
As you said SVD can be used to factorize a full matrix without missing value. For sparse matrices we minimize the least square problem to get vector representing latent factors of an item 'qi' and a user 'pu'. You can start with some random values for the above latent factors and update them to get factors that minimizes the least square problem. This is your simple SVD in context with recommender system. SVD++ improves things over SVD by involving user implicit feedback which is termed as 'yi' and this helps you to characterize users based on set of items that they have rated.
