Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Wikipedia states (link below) that by the Eckart-Young-Mirsky theorem, the SVD provides the best low rank matrix approximation (on the basis of Frobenius norm of the error matrix) for any matrix A in R^m*n where m>=n.
Does this mean that the SVD is only proven to be the best approximation for over determined matrices? Does the proof not hold or is there no proof for under determined matrices with more columns than rows?
If we start from the premise that we know how to do the SVD of $A$ for $A$ with dimension $n \times m$ and $m < n$, then it's simple enough to write down the SVD of $A^\top$ using only the properties of matrix transpose. $$ \begin{align} A &= USV^\top \\ A^\top &= V S U^\top \end{align} $$
Because the result is also an SVD, we know that it is the best low-rank approximation to $A$ in the sense of the Eckart-Young-Mirsky theorem.
