Similar questions have been asked a lot of times but I have not found an answer that gives an intuitive explanation as to why this works.
For reference I have read the answers here and here.
As I understand it, when we are given a matrix with missing values, one way to predict those values is the following:
- Fill the missing values with 0s or average of the column or row etc.
- Calculate the SVD.
- Set low singular values to 0.
- Reconstruct the matrix(which is now a low-rank approximation)
I don't quite follow why this procedure works. Eckart-Young theorem says that this is the optimal solution in terms of minimizing the Frobenius norm.
However, to me it seems that this procedure will minimize the Frobenius norm between the low-rank matrix and the matrix with the imputed values. So the prediction of a missing cell of the matrix, will just be a number very close the value I inserted before running the SVD.(if I were to keep all singular values I would get the exact matrix back with the values I inserted so I would get no helpful predictions.)
So how is the accuracy of my predictions improved by removing singular values?
(I am aware there are better methods to do matrix completion but for now I am interested in this one)
