1
$\begingroup$

I have computed a correlation matrix from certain data set of dimension 6

The correlation matrix is:

 1.000000000000000 -0.142753907555000 -0.192138186332000 -0.523853268770000  0.124444699394000  0.002606132276000
-0.142753907555000  1.000000000000000  0.035741740609000  0.052631092693000  0.017746700663000 -0.361033166149000
-0.192138186332000  0.035741740609000  1.000000000000000  0.119582107782000 -0.346476373927000 -0.213738934536000
-0.523853268770000  0.052631092693000  0.119582107782000  1.000000000000000 -0.128989440426000  0.048428923563000
 0.124444699394000  0.017746700663000 -0.346476373927000 -0.128989440426000  1.000000000000000 -0.434021957325000
 0.002606132276000 -0.361033166149000 -0.213738934536000  0.048428923563000 -0.434021957325000  1.000000000000000

Then I applied SVD and got:

$U$

-0.568204264304213  0.191580010050053  0.344459905939842  0.057623813300714 -0.711866787458235  0.108229201374774
 0.092203508694887 -0.529503080968478  0.114920527567713  0.783048200494938 -0.053598897055946  0.286148469345589
 0.399921613786302 -0.136693414712789  0.669267980471650 -0.404735519465042  0.004693646229437  0.457848350968476
 0.545763234721263 -0.103897122514854 -0.450027359344472 -0.130624377930601 -0.685583680847584  0.041659858939764
-0.440777193839364 -0.406362489876042 -0.425730028151024 -0.360866606588038  0.093306539446761  0.566046038136009
 0.128697656758384  0.698794255931705 -0.190919707811538  0.269103577206925  0.107794763719302  0.612075746815743

$S$ 

1.757508326665900 0.000000000000000 0.000000000000000 0.000000000000000 0.000000000000000 0.000000000000000
0.000000000000000 1.561285064351230 0.000000000000000 0.000000000000000 0.000000000000000 0.000000000000000
0.000000000000000 0.000000000000000 1.108208836182120 0.000000000000000 0.000000000000000 0.000000000000000
0.000000000000000 0.000000000000000 0.000000000000000 0.829989535414321 0.000000000000000 0.000000000000000
0.000000000000000 0.000000000000000 0.000000000000000 0.000000000000000 0.469300557533083 0.000000000000000
0.000000000000000 0.000000000000000 0.000000000000000 0.000000000000000 0.000000000000000 0.273707679853350

$V^T$

-0.568204264304213  0.092203508694887  0.399921613786303  0.545763234721263 -0.440777193839364  0.128697656758384
 0.191580010050053 -0.529503080968478 -0.136693414712789 -0.103897122514854 -0.406362489876042  0.698794255931705
 0.344459905939843  0.114920527567713  0.669267980471650 -0.450027359344472 -0.425730028151024 -0.190919707811538
 0.057623813300715  0.783048200494939 -0.404735519465042 -0.130624377930600 -0.360866606588038  0.269103577206925
-0.711866787458234 -0.053598897055946  0.004693646229437 -0.685583680847584  0.093306539446760  0.107794763719302
 0.108229201374774  0.286148469345589  0.457848350968475  0.041659858939764  0.566046038136010  0.612075746815743

Now what do I do with this decomposition, Maybe i am wrong but $S$ has the eigenvalues, and $U$ has the eigenvectors. So how can I reduce dimension in this example? How would you interpret these results? Can you give a little guidance?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Component scores = standardized data * U $\endgroup$
    – ttnphns
    Commented Nov 16, 2012 at 19:21
  • $\begingroup$ So I would take my original dataset and multiply it by $U$? and then sort the results, and the one with less value I can dismiss? $\endgroup$
    – edgarmtze
    Commented Nov 16, 2012 at 19:27
  • 1
    $\begingroup$ No sort needed because U's columns correspond to the eigenvalues which are sorted desdendingly. Just multiply Z*U and then keep first m columns of the result. Here you are with data reduced to m components. $\endgroup$
    – ttnphns
    Commented Nov 16, 2012 at 19:34

1 Answer 1

Reset to default
4
$\begingroup$

First of all, since your correlation matrix is a square matrix you may want to do Principle Component Analysis (PCA), I guess doing SVD on a square matrix is same as doing PCA, so you should be fine.

I assume your original aim is to reduce data dimensionality. With whatever information you have provided, I see that you are working with only 6 features and for most practical purposes you can keep working with original 6 features without worrying about omitting some of them.

If your aim is to map original data onto new space where features are non-correlated, then PCA can help. The eigenvectors that you have obtained above denote new axes (they are orthogonal to each other) and corresponding eigenvalues are indicators of how much variance in the data is accounted by each axes (once you have accounted for previous axes).

You can do the following: You can set a threshold, say 80%, select appropriate number of eigenvalues so that 80% of the energy (or variance) is accounted for. In your case most likely first 4 (out of 6) would suffice. Then you project your data onto first 4 new axes (defined by first 4 eigenvectors) and then take your problem forward from that point.

PS: A good resource to learn more about LDA/LSI is Chapter 18 of freely available book Introduction to Information Retrieval.

Improve this answer
$\endgroup$
2
  • $\begingroup$ You said the first 4 because their $S$ values are $1.75$, $1.56$, $1.10$ and $0.82$ the others I can dismiss are $0.46$ and $0.27$ is that correct? the same goes for $U$? One more, so which attribute I am dropping $5$ and $6$? or how do you know that? $\endgroup$
    – edgarmtze
    Commented Nov 16, 2012 at 19:42
  • $\begingroup$ @cMinor: First of all you need to decide your threshold. I did not do explicit calculations, but if eigenvalues are sorted in descending order and cumulative sum of first m eigenvalues is 80% of their cumulative sum, then keep only first m columns in U, m x m submatrix of S and m rows of V'. $\endgroup$
    – abhinavkulkarni
    Commented Nov 16, 2012 at 19:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.