# What do the matrix (S, U, V) returned by singular value decomposition represent (in terms of variation)?

I believe SVD on a matrix A returns three matrices: U, S, and V.

Let's imagine A is a data matrix with training examples/records/whatever you call them as its rows and attributes as its columns.

I think S is a diagonal matrix, where the $$i$$-th diagonal value is the variation in the $$i$$-th attribute (column) of the matrix A. Furthermore, the diagonal values of S decrease as you go left to right/top to bottom (the matrix is sorted).

I think U says something about the records themselves. I believe each row represents one record. I often see the first two columns U graphed such that the x axis is U1 (the first column) and the y axis is U2, but I don't know what the resulting graph is telling us.

I haven't been able to figure out what V does.

Is my understanding of S correct? And what do U and V represent? Any help is appreciated!

By definition: Given that an $$m\times n$$ matrix $$A$$ has rank $$r$$, $$A$$ can be factored $$A=U\times S\times V^T$$, where $$U$$ and $$V$$ are orthogonal matrices containing the singular vectors. We can think of $$U$$ and $$V$$ as rotations and reflections and $$S$$ as the stretching matrix. Since $$V$$ is an orthogonal matrix $$(𝐕^⊤𝐕=𝐈)$$, $$AA^T=(USV^T)(VSU^T)=USU^T$$, where $$S$$ has all the eigenvalues, and $$U$$ has its eigenvectors.