2
$\begingroup$

I have a data matrix $X$ and I perform a PCA on this data with:

%// Subtract the mean from the data
Y = bsxfun(@minus, X, mean(X));

%// Obtain the PCA solution by calculate the SVD of Y
[U, S, V] = svd(Y);

%// Compute variance explained
rho = diag(S).^2./sum(diag(S).^2);

%// Plot variance explained
plot(rho, 'o-')
title('Variance explained by principal components');
xlabel('Principal component');
ylabel('Variance explained');

I get this plot

enter image description here

What is the reason for this to happen? It cannot be correct that only the first PC can explain all the variance.

My data looks like

price       crime      nox        rooms      dist          radial  proptax    stratio   lowstat   lprice     lnox      lproptax
==========================================================================================================================
24000       .006       5.38       6.57       4.09          1       29.6       15.3       4.98   10.08581   1.682688    5.69036  
21599       .027       4.69       6.42       4.97          2       24.2       17.8       9.14   9.980402   1.545433   5.488938  
34700       .027       4.69       7.18       4.97          2       24.2       17.8       4.03    10.4545   1.545433   5.488938  
33400       .032       4.58          7       6.06          3       22.2       18.7       2.94   10.41631   1.521699   5.402678  
...

so the values of first attribute are much higher than the rest. Can this be the reason to the weird "variance explained" plot?

Improve this question
$\endgroup$
7
  • 3
    $\begingroup$ Yes. The "total variance" of your dataset is simply the sum of the variances of all the variables. Your price variable will dominate the sum, and hence the PC1 will essentially equal to price and will explain almost all the variance. You probably want to run PCA on correlations not on covariances (see stats.stackexchange.com/questions/53), i.e. you probably should standardize your data after centering but prior to doing SVD: Y = bsxfun(@times, Y, 1./std(Y)); $\endgroup$
    – amoeba
    Sep 28, 2015 at 16:52
  • 2
    $\begingroup$ No, that's just centering. The variances can still be very different, check for yourself by running var(Y). $\endgroup$
    – amoeba
    Sep 28, 2015 at 16:55
  • 1
    $\begingroup$ PC1 is just a linear combination of your original variables. Of course you can see how they are related. If you don't standardize, it will be $\sim1$ times price plus $\sim 0$ times everything else. $\endgroup$
    – amoeba
    Sep 28, 2015 at 17:00
  • 2
    $\begingroup$ See stats.stackexchange.com/questions/87037/… for another example in which uncritical use of covariance matrix produced garbage, which was not even noticed by all contributors to the thread. $\endgroup$
    – Nick Cox
    Sep 28, 2015 at 17:11
  • 1
    $\begingroup$ After some reflection, I edited the top answer in the "PCA on correlation or covariance" thread (to include some plots) and I think this question is now a duplicate of that one. In the dataset considered there PCA on covariance explains over 98% of the variance. $\endgroup$
    – amoeba
    Sep 29, 2015 at 11:31

0

Reset to default

Browse other questions tagged or ask your own question.