In practice, what do people do with the missing values from the recommendation matrix, which is the whole point of doing the calculation? My guess from reading Simon's blog post is that he ONLY uses the non-missing terms to build a model.
That's right -- that's the point of his and your model, to predict the missing terms, right? It's a crucial point that many actually forget. They think they can just "assume" to pre-assign a constant to missing data without a care in the world, and things will magically work out well enough from an SVD. Garbage in, garbage out: It's real, and you had better watch it. You'd better not feed junk data to a model if you want something useful to result.
It is certainly NOT "best to infer any missing values" on majority sparse dataset and then run SVD on that with some hope to impute values for you (which you already imputed before you ran SVD, right?). What do you think, a model is magic? THere is no magic nor technology to overcome majority garbage data. You cannot lie to a model that data is real data when it's not real at all, but really just some junk you just plain made up from thin air.
SVD does other useful things so I'm certainly not saying SVD is worthless in the least. Go ahead and use SVD only on complete datasets, perhaps which you've intelligently imputed missing values on already using a machine learning model with all due attention to bias error and variance error during its development.
Machine learning is the way. So if you still want to know how to impute values using a matrix factorization design, there are certainly good ways to do exactly this using machine learning, and importantly they don't feed any junk data to a model to pointlessly attempt to learn from.
Exactly such a machine learning matrix factorization model is presented quite well by the instructors of the Stanford online course Mining Massive Data Sets, in module 5. They show you the math and explain the model. They don't code it up for you though.
It's OK because you can code it up yourself, if you understand basic machine learning. Do you know what a loss function and a cost function are? Regularization? Gradient descent? ARe you OK with matrix multiplication and addition? Bias error and variance error? If so then you are good. If not then you should consider to take Andrew Ng's online course Machine Learning at Coursera, which is one of the many good starting places. Then also go take the online course Mining Massive Data Sets which talks exactly about matrix factorization and machine learning for making recommender models.
Suffice it to say, you can completely design as well as code up your own factorization model which handles missing data very well, just like Simon Funk did, and you can do it from scratch but it's not hard at all any more like it was back in his day, because now you can use a tool like TensorFlow or Microsoft CNTK which does a lot for you. Define a loss function and a cost function, choose an optimizer, partition your dataset into training, dev, test from the data that's actually available (labeled data) and let it run. Seriously, it works. It's not easy debugging TF and its graph building errors, but it can work great in the end and takes less than one page of code.
Specifically, one way to not feed fake data to a matrix factorization machine learning model, is to skip over the missing data's matrix elements in your loss and cost functions.