3
$\begingroup$

I am confused by the singular value decomposition of a matrix. This may just be a misunderstanding of what singular value decomposition does so please be gentle with me. If I do a singular value decomposition on $X$ with $m$ rows and $n$ columns such that I get $X=U\Sigma V$. $V^*$ is supposed to be a nxn square matrix (see e.g. Wikipedia. However, for a matrix x e.g. size (40,100) I get in Julia (and also in R):

x = randn(40, 100)
xsvd = svdfact(x)
size(xsvd.Vt)
(40,100)

I am expecting (100,100). However, for 

x = randn(100, 40)
xsvd = svdfact(x)
size(xsvd.Vt)
(40,40)

I get what I expect.

Can someone please explain to me what is going on here? And possibly point me to somewhere where I can read up on the fundamentals?

Improve this question
$\endgroup$
4
  • $\begingroup$ When you do SVD of a nXp matrix, you normally get 3 matrices as the result: left eigenvectors U (nXn), right eigenvectors V (pXp), diagonal matrix of singular values S (nXp). Some implementations of the function may cut-off empty rows or columns of S. $\endgroup$
    – ttnphns
    Commented May 6, 2014 at 11:02
  • $\begingroup$ @ttnphns: yes you are right. My question is with respect to the right eigenvectors $V^*$. I should have been more precise I guess. $\endgroup$
    – joidegn
    Commented May 6, 2014 at 11:42
  • $\begingroup$ You should take into account that only eigenvectors corresponding to nonzero singular values make sense. So, the trailing eigenvectors of V, corresponding to zero singular values, can be safely set to zero. If so, they may be not computed or not shown by a function. $\endgroup$
    – ttnphns
    Commented May 6, 2014 at 13:17
  • $\begingroup$ you changed the first comment. The second one is similar to what I accepted as the right answer by Brian Borchers below. Thanks $\endgroup$
    – joidegn
    Commented May 8, 2014 at 11:08

2 Answers 2

Reset to default
6
$\begingroup$

If you look at $U \Sigma V^{T}$, when $m<n$, you will see that the last $n-m$ columns of $V$ (rows of $V^{T}$) are multiplied by 0. Many software packages compute a compact form of the SVD in which these columns are not returned.

Improve this answer
$\endgroup$
2
  • $\begingroup$ So the singular values in $\Sigma$ are 0? That would make some sense. Now I know what happens. Can you explain to me or point me to somewhere where I can learn why that happens? $\endgroup$
    – joidegn
    Commented May 6, 2014 at 14:54
  • $\begingroup$ $\Sigma$ is an $m$ by $n$ diagonal matrix, with possibly nonzero singular values in $\Sigma_{1,1}$ through $\sigma_{m,m}$, but the singular values for $m+1$, $\ldots$, $n$ are all considered to be 0 (there's no place to put these zeros in the $m$ by $n$ matrix $\Sigma$.) $\endgroup$
    – Brian Borchers
    Commented May 6, 2014 at 15:57
0
$\begingroup$

I faced this same problem in SciDart (https://scidart.org/) and the answer of Brian Borchers helped me find an away out. But my approach was different (you can find my implementation here https://github.com/scidart/scidart/blob/9f5d4e42996b35d32976d571b2acd8804298ed4d/lib/src/numdart/linalg/decompositions/svd.dart#L84 ) and my implementation is based on JAMA library (https://math.nist.gov/javanumerics/jama/doc/Jama/SingularValueDecomposition.html):

  1. if $m < n$, add new $n - m$ lines to your input and fill it with zeros, you will get a $n x n$ square matrix as input;
  2. compute de SVD;
  3. if $m < n$, multiply each element of $U$ and $V$ by -1;

Where

  • $m$ is the number of rows;
  • $n$ is the number of columns;
Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Could you explain what the multiplication by $-1$ is intended to do? It does not appear to alter any of the defining properties of the SVD. $\endgroup$
    – whuber ♦
    Commented May 17, 2021 at 21:40
  • $\begingroup$ I can’t explain mathematically but It’s probably a artifact of the algorithm that I’m using. $\endgroup$
    – Ângelo Polotto
    Commented Aug 21, 2021 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.