0
$\begingroup$

I believe SVD on a matrix A returns three matrices: U, S, and V.

Let's imagine A is a data matrix with training examples/records/whatever you call them as its rows and attributes as its columns.

I think S is a diagonal matrix, where the $i$-th diagonal value is the variation in the $i$-th attribute (column) of the matrix A. Furthermore, the diagonal values of S decrease as you go left to right/top to bottom (the matrix is sorted).

I think U says something about the records themselves. I believe each row represents one record. I often see the first two columns U graphed such that the x axis is U1 (the first column) and the y axis is U2, but I don't know what the resulting graph is telling us.

I haven't been able to figure out what V does.

Is my understanding of S correct? And what do U and V represent? Any help is appreciated!

$\endgroup$
0
$\begingroup$

By definition: Given that an $m\times n$ matrix $A$ has rank $r$, $A$ can be factored $A=U\times S\times V^T$, where $U$ and $V$ are orthogonal matrices containing the singular vectors. We can think of $U$ and $V$ as rotations and reflections and $S$ as the stretching matrix. Since $V$ is an orthogonal matrix $(𝐕^⊤𝐕=𝐈)$, $AA^T=(USV^T)(VSU^T)=USU^T$, where $S$ has all the eigenvalues, and $U$ has its eigenvectors.

If you really wonder what U, S, and V do for the A see the following figure from http://web.cs.iastate.edu/~cs577/handouts/svd.pdfenter image description here

$\endgroup$
  • $\begingroup$ Indeed, there are many many interpretations of SVD explained by linear algebra alone. But there is a non-mathematical understanding of the matrices S, U, V that I know are related to statistical variation among attributes/training examples for a data matrix A; this interpretation is what makes it useful in the context of machine learning. $\endgroup$ – James Ronald Nov 9 at 18:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.