1
$\begingroup$

I am confused by the singular value decomposition of a matrix. This may just be a misunderstanding of what singular value decomposition does so please be gentle with me. If I do a singular value decomposition on $X$ with $m$ rows and $n$ columns such that I get $X=U\Sigma V$. $V^*$ is supposed to be a nxn square matrix (see e.g. Wikipedia. However, for a matrix x e.g. size (40,100) I get in Julia (and also in R):

x = randn(40, 100)
xsvd = svdfact(x)
size(xsvd.Vt)
(40,100)

I am expecting (100,100). However, for

x = randn(100, 40)
xsvd = svdfact(x)
size(xsvd.Vt)
(40,40)

I get what I expect.

Can someone please explain to me what is going on here? And possibly point me to somewhere where I can read up on the fundamentals?

$\endgroup$
4
  • $\begingroup$ When you do SVD of a nXp matrix, you normally get 3 matrices as the result: left eigenvectors U (nXn), right eigenvectors V (pXp), diagonal matrix of singular values S (nXp). Some implementations of the function may cut-off empty rows or columns of S. $\endgroup$ – ttnphns May 6 '14 at 11:02
  • $\begingroup$ @ttnphns: yes you are right. My question is with respect to the right eigenvectors $V^*$. I should have been more precise I guess. $\endgroup$ – joidegn May 6 '14 at 11:42
  • $\begingroup$ You should take into account that only eigenvectors corresponding to nonzero singular values make sense. So, the trailing eigenvectors of V, corresponding to zero singular values, can be safely set to zero. If so, they may be not computed or not shown by a function. $\endgroup$ – ttnphns May 6 '14 at 13:17
  • $\begingroup$ you changed the first comment. The second one is similar to what I accepted as the right answer by Brian Borchers below. Thanks $\endgroup$ – joidegn May 8 '14 at 11:08
4
$\begingroup$

If you look at $U \Sigma V^{T}$, when $m<n$, you will see that the last $n-m$ columns of $V$ (rows of $V^{T}$) are multiplied by 0. Many software packages compute a compact form of the SVD in which these columns are not returned.

$\endgroup$
2
  • $\begingroup$ So the singular values in $\Sigma$ are 0? That would make some sense. Now I know what happens. Can you explain to me or point me to somewhere where I can learn why that happens? $\endgroup$ – joidegn May 6 '14 at 14:54
  • $\begingroup$ $\Sigma$ is an $m$ by $n$ diagonal matrix, with possibly nonzero singular values in $\Sigma_{1,1}$ through $\sigma_{m,m}$, but the singular values for $m+1$, $\ldots$, $n$ are all considered to be 0 (there's no place to put these zeros in the $m$ by $n$ matrix $\Sigma$.) $\endgroup$ – Brian Borchers May 6 '14 at 15:57
0
$\begingroup$

I faced this same problem in SciDart (https://scidart.org/) and the answer of Brian Borchers helped me find an away out. But my approach was different:

  1. if $m < n$, add new $n - m$ lines to your input and fill it with zeros, you will get a $n x n$ square matrix as input;
  2. compute de SVD;
  3. if $m < n$, multiply each element of $U$ and $V$ by -1;

Where

  • $m$ is the number of rows;
  • $n$ is the number of columns;
$\endgroup$
1
  • 1
    $\begingroup$ Could you explain what the multiplication by $-1$ is intended to do? It does not appear to alter any of the defining properties of the SVD. $\endgroup$ – whuber May 17 at 21:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.