There are lots of ways to predict missing values but classic SVD is not one of them. The nice thing is that machine learning now offers many ways to do this, some of which are based on matrix factorization, others completely different than matrix factorization. You can choose and make a completely custom model, and this is commonly done now because the tools are powerful enough today. Matrix factorization is still certainly a good way to predict missing values in sparse data, but SVD itself is not.
The accepted answer here, apparently advised the questioner to just pick any constant value such as 0 or 99 or -3 or whatever, to assign to the missing values in advance, and then run SVD on that. This is a bad answer if the goal is to predict on sparse datasets. But if instead the OP's goal is simply to run SVD, then pre-assigning any constant value will work fine, so pick any value and then run SVD if the results do not matter for the OP. I said SVD is a bad solution for prediction of missing values because assuming a constant value in all the sparse locations could end up being that you introducing literally more noise points than known good data points.
What's the point of learning noise? And why would you even suggest that the missing values are actually the same constant value, when the point of the exercise is to predict what they are? You don't expect the missing values to really be all the same, right? That's going to underestimate the number of principal components that result if there's constant data so pervasive in your dataset, for one thing. Also that's a very easy prediction problem then. You don't need a learning algorithm or even a factorization algorithm. You just said the missing values are a known constant. No need to impute! You did that already, manually, by just guessing the old fashioned way.
You can get fancier with SVD and pre-impute the missing values using a random distribution that's empirically derived using the mean and standard deviation from the known (non-missing) data. But then there's randomness instead of patterns in the data and you presumably expected matrix factorization and dimensionality reduction inherent in that technique to find the patterns that you expect are there. You won't discover many patterns of any use in random noise, though, so it's not helping to use this way either.
The bottom line is that the output of SVD -- or any other algorithm -- will be largely garbage whenever there is an overwhelming amount of investigator-provided junk data fed in. No algorithm can learn a good model from majority junk data. Just say no to that whole "approach."
It seems likely that the OP's goal is to predict, and to use a matrix factorization design as part of the learning algorithm. In this case, the nice thing is you can feasibly write your own cost function which crucially omits from the cost, any predictions made against the missing values. No junk data whatsoever is fed to the learning algorithm this way. Use a good gradient-descent based optimizer, such as Adam (there are others). You can get a solution that's measurably accurate to whatever degree on training, dev, and test dataset, provided you follow a good machine learning project methodology. Feel free to add terms and complexity to your model like user bias, item bias, global bias, regularization, or whatever else you need to control bias error and variance error to your project's requirements and available datasets.
A modern machine learning development package make this a practical approach now. TensorFlow for example (or Microsoft CNTK et al) can help you do exactly what I described on a sparse dataset using a matrix factorization model.
