Skip to main content
Cross Validated

Return to Answer

added 1011 characters in body
Source Link
ttnphns
ttnphns
  • 58.8k
  • 53
  • 286
  • 511

Please note also a particular situation of singularity. If there are collinearities (co-proportionalities) in your matrix X one or more last singular values will be zero and D matrix might look like this, for example:

D: one zero singular value in 4-column [min(nrows,ncols) in X] D
   because of a singularity in X
   3.781503197    .000000000    .000000000    .000000000
    .000000000   2.427179503    .000000000    .000000000
    .000000000    .000000000   1.762938484    .000000000
    .000000000    .000000000    .000000000    .000000000
    .000000000    .000000000    .000000000    .000000000
    .000000000    .000000000    .000000000    .000000000

In such a case, you are in right to cut D to size 3 x 3 because only
3 singular values are "real" or tangible dimensions of the data.

Then, accordingly you will leave just 3 first columns in U and in V.
And you will successfully restore your X data with these truncated matrices.

Please note also a particular situation of singularity. If there are collinearities (co-proportionalities) in your matrix X one or more last singular values will be zero and D matrix might look like this, for example:

D: one zero singular value in 4-column [min(nrows,ncols) in X] D
   because of a singularity in X
   3.781503197    .000000000    .000000000    .000000000
    .000000000   2.427179503    .000000000    .000000000
    .000000000    .000000000   1.762938484    .000000000
    .000000000    .000000000    .000000000    .000000000
    .000000000    .000000000    .000000000    .000000000
    .000000000    .000000000    .000000000    .000000000

In such a case, you are in right to cut D to size 3 x 3 because only
3 singular values are "real" or tangible dimensions of the data.

Then, accordingly you will leave just 3 first columns in U and in V.
And you will successfully restore your X data with these truncated matrices.
Source Link
ttnphns
ttnphns
  • 58.8k
  • 53
  • 286
  • 511

I think that this question has been already covered on the site. I'm answering via an example. Both your citations are correct.

X
   -.910218691   -.037212328   -.275548166   -.360441580
  -1.855949714  -1.767057411   -.317283890   1.633887711
   -.186585570   -.316960442   -.340387838  -1.160337400
   1.427254188   -.872044568   -.447456104  -1.785203938
  -1.524117328   1.013061127    .735027680    .583921184
    .014211669   -.880275500   1.516716199   -.266348030

svd(x) = UDV'

D
   3.781503197    .000000000    .000000000    .000000000
    .000000000   2.427179503    .000000000    .000000000
    .000000000    .000000000   1.762938484    .000000000
    .000000000    .000000000    .000000000   1.494400129
    .000000000    .000000000    .000000000    .000000000
    .000000000    .000000000    .000000000    .000000000

U
  -.1045359966   .0613688431   .1533784675   .5925648676  -.7620680911  -.1729168539
  -.6655219390   .7063271894   .1384176823  -.0473139686   .1623946521  -.1020409758
   .1754941747   .1964628810   .0893095485   .6365658833   .3639134791   .6204665918
   .5934543082   .4282595430   .0031973635   .1744522991   .2294862700  -.6174932972
  -.4039262123  -.4669763619  -.2167568310   .4571230653   .4262068341  -.4256517430
  -.0031887094   .2393009491  -.9499187589   .0449114759  -.1616507584   .1105948958

V
   .7299152783  -.0317492361  -.0520390915  -.6808139286
   .0529861206  -.9763855955   .1901410033   .0878068268
  -.1023534922  -.1976805415  -.9745608685  -.0260245989
  -.6737506364  -.0811514539   .1066276016  -.7267083844

UDV' restores X
   -.910218691   -.037212328   -.275548166   -.360441580
  -1.855949714  -1.767057411   -.317283890   1.633887711
   -.186585570   -.316960442   -.340387838  -1.160337400
   1.427254188   -.872044568   -.447456104  -1.785203938
  -1.524117328   1.013061127    .735027680    .583921184
    .014211669   -.880275500   1.516716199   -.266348030

-----------------

Observe that (because X wasn't square) D above is not square.
The singular values are only four, the min(nrows,ncols) in X.
So, cut empty rows (or columns) in D so it becomes square, 4 x 4.
D
   3.781503197    .000000000    .000000000    .000000000
    .000000000   2.427179503    .000000000    .000000000
    .000000000    .000000000   1.762938484    .000000000
    .000000000    .000000000    .000000000   1.494400129

Then leave only 4 first columns ("real" left eigenvectors) in U
U
  -.1045359966   .0613688431   .1533784675   .5925648676
  -.6655219390   .7063271894   .1384176823  -.0473139686
   .1754941747   .1964628810   .0893095485   .6365658833
   .5934543082   .4282595430   .0031973635   .1744522991
  -.4039262123  -.4669763619  -.2167568310   .4571230653
  -.0031887094   .2393009491  -.9499187589   .0449114759

And only 4 first columns ("real" right eigenvectors) in V
(in this example, V was initially 4-column, so leave as is)
V
   .7299152783  -.0317492361  -.0520390915  -.6808139286
   .0529861206  -.9763855955   .1901410033   .0878068268
  -.1023534922  -.1976805415  -.9745608685  -.0260245989
  -.6737506364  -.0811514539   .1066276016  -.7267083844

UDV' restores X as well as it was above
   -.910218691   -.037212328   -.275548166   -.360441580
  -1.855949714  -1.767057411   -.317283890   1.633887711
   -.186585570   -.316960442   -.340387838  -1.160337400
   1.427254188   -.872044568   -.447456104  -1.785203938
  -1.524117328   1.013061127    .735027680    .583921184
    .014211669   -.880275500   1.516716199   -.266348030