1
$\begingroup$

I have computed a correlation matrix from certain data set of dimension 6

The correlation matrix is:

 1.000000000000000 -0.142753907555000 -0.192138186332000 -0.523853268770000  0.124444699394000  0.002606132276000
-0.142753907555000  1.000000000000000  0.035741740609000  0.052631092693000  0.017746700663000 -0.361033166149000
-0.192138186332000  0.035741740609000  1.000000000000000  0.119582107782000 -0.346476373927000 -0.213738934536000
-0.523853268770000  0.052631092693000  0.119582107782000  1.000000000000000 -0.128989440426000  0.048428923563000
 0.124444699394000  0.017746700663000 -0.346476373927000 -0.128989440426000  1.000000000000000 -0.434021957325000
 0.002606132276000 -0.361033166149000 -0.213738934536000  0.048428923563000 -0.434021957325000  1.000000000000000

Then I applied SVD and got:

$U$

-0.568204264304213  0.191580010050053  0.344459905939842  0.057623813300714 -0.711866787458235  0.108229201374774
 0.092203508694887 -0.529503080968478  0.114920527567713  0.783048200494938 -0.053598897055946  0.286148469345589
 0.399921613786302 -0.136693414712789  0.669267980471650 -0.404735519465042  0.004693646229437  0.457848350968476
 0.545763234721263 -0.103897122514854 -0.450027359344472 -0.130624377930601 -0.685583680847584  0.041659858939764
-0.440777193839364 -0.406362489876042 -0.425730028151024 -0.360866606588038  0.093306539446761  0.566046038136009
 0.128697656758384  0.698794255931705 -0.190919707811538  0.269103577206925  0.107794763719302  0.612075746815743

$S$

1.757508326665900 0.000000000000000 0.000000000000000 0.000000000000000 0.000000000000000 0.000000000000000
0.000000000000000 1.561285064351230 0.000000000000000 0.000000000000000 0.000000000000000 0.000000000000000
0.000000000000000 0.000000000000000 1.108208836182120 0.000000000000000 0.000000000000000 0.000000000000000
0.000000000000000 0.000000000000000 0.000000000000000 0.829989535414321 0.000000000000000 0.000000000000000
0.000000000000000 0.000000000000000 0.000000000000000 0.000000000000000 0.469300557533083 0.000000000000000
0.000000000000000 0.000000000000000 0.000000000000000 0.000000000000000 0.000000000000000 0.273707679853350

$V^T$

-0.568204264304213  0.092203508694887  0.399921613786303  0.545763234721263 -0.440777193839364  0.128697656758384
 0.191580010050053 -0.529503080968478 -0.136693414712789 -0.103897122514854 -0.406362489876042  0.698794255931705
 0.344459905939843  0.114920527567713  0.669267980471650 -0.450027359344472 -0.425730028151024 -0.190919707811538
 0.057623813300715  0.783048200494939 -0.404735519465042 -0.130624377930600 -0.360866606588038  0.269103577206925
-0.711866787458234 -0.053598897055946  0.004693646229437 -0.685583680847584  0.093306539446760  0.107794763719302
 0.108229201374774  0.286148469345589  0.457848350968475  0.041659858939764  0.566046038136010  0.612075746815743

Now what do I do with this decomposition, Maybe i am wrong but $S$ has the eigenvalues, and $U$ has the eigenvectors. So how can I reduce dimension in this example? How would you interpret these results? Can you give a little guidance?

$\endgroup$
  • 1
    $\begingroup$ Component scores = standardized data * U $\endgroup$ – ttnphns Nov 16 '12 at 19:21
  • $\begingroup$ So I would take my original dataset and multiply it by $U$? and then sort the results, and the one with less value I can dismiss? $\endgroup$ – cMinor Nov 16 '12 at 19:27
  • 1
    $\begingroup$ No sort needed because U's columns correspond to the eigenvalues which are sorted desdendingly. Just multiply Z*U and then keep first m columns of the result. Here you are with data reduced to m components. $\endgroup$ – ttnphns Nov 16 '12 at 19:34
3
$\begingroup$

First of all, since your correlation matrix is a square matrix you may want to do Principle Component Analysis (PCA), I guess doing SVD on a square matrix is same as doing PCA, so you should be fine.

I assume your original aim is to reduce data dimensionality. With whatever information you have provided, I see that you are working with only 6 features and for most practical purposes you can keep working with original 6 features without worrying about omitting some of them.

If your aim is to map original data onto new space where features are non-correlated, then PCA can help. The eigenvectors that you have obtained above denote new axes (they are orthogonal to each other) and corresponding eigenvalues are indicators of how much variance in the data is accounted by each axes (once you have accounted for previous axes).

You can do the following: You can set a threshold, say 80%, select appropriate number of eigenvalues so that 80% of the energy (or variance) is accounted for. In your case most likely first 4 (out of 6) would suffice. Then you project your data onto first 4 new axes (defined by first 4 eigenvectors) and then take your problem forward from that point.

PS: A good resource to learn more about LDA/LSI is Chapter 18 of freely available book Introduction to Information Retrieval.

$\endgroup$
  • $\begingroup$ You said the first 4 because their $S$ values are $1.75$, $1.56$, $1.10$ and $0.82$ the others I can dismiss are $0.46$ and $0.27$ is that correct? the same goes for $U$? One more, so which attribute I am dropping $5$ and $6$? or how do you know that? $\endgroup$ – cMinor Nov 16 '12 at 19:42
  • $\begingroup$ @cMinor: First of all you need to decide your threshold. I did not do explicit calculations, but if eigenvalues are sorted in descending order and cumulative sum of first m eigenvalues is 80% of their cumulative sum, then keep only first m columns in U, m x m submatrix of S and m rows of V'. $\endgroup$ – abhinavkulkarni Nov 16 '12 at 19:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.