2
$\begingroup$

I try to implement PCA in MATLAB using the Fisher iris data. I am having a problem finding out the percentage of variance in Principal Component Analysis.

Running this code

load fisheriris
X = bsxfun(@minus, meas, mean(meas));
[pc, ~, latent] = pca(X)
cumsum(latent)/sum(latent)

I get the following result:

pc =

    0.3614    0.6566   -0.5820    0.3155
   -0.0845    0.7302    0.5979   -0.3197
    0.8567   -0.1734    0.0762   -0.4798
    0.3583   -0.0755    0.5458    0.7537


latent =

    4.2282
    0.2427
    0.0782
    0.0238


ans =

    0.9246
    0.9777
    0.9948
    1.0000

Now how do you choose the percentage of variance? (In the "hald" dataset it is 98%, but I also don't understand why 98%.)

$\endgroup$
3
  • $\begingroup$ Do you mean the percentage of variance explained by each component? Do you know about the R2 statistic? What do you mean with "hald" dataset? $\endgroup$ Jan 9, 2014 at 20:58
  • $\begingroup$ Can you give more description, or a label, to each of these sets of statistics. I know about PCA, but I don't know what matlab calls them. $\endgroup$ Jan 9, 2014 at 21:06
  • $\begingroup$ I mean "ingredients" database in hald. Example is given here: mathworks.com/help/stats/princomp.html I don't understand from where the 98% comes: "The following command and plot show that two components account for 98% of the variance" , thank you in advance! $\endgroup$
    – user19565
    Jan 11, 2014 at 23:38

1 Answer 1

6
$\begingroup$

The Eigenvalues tell you this for each component. If you sum the Eigenvalues you get the total variance in the data. You can express the Eigenvalue as a proportion of variance explained by that component via

$$ \frac{\lambda_i}{\sum_{i = 1}^m \lambda_i} $$

Where $\lambda_i$ is the Eigenvalue for the $i$th component and $m$ the number of variables in the input data.

Using R with Edgar Anderson's (:P) Iris data, we do

pca <- prcomp(iris[, -5])

As this uses a SVD to do the PCA and reports the singular values, the square roots of the Eigenvalues, in component $sdev and hence we must square them:

R> pca$sdev^2 / sum(pca$sdev^2)
[1] 0.924619 0.053066 0.017103 0.005212
R> cumsum(pca$sdev^2 / sum(pca$sdev^2))
[1] 0.9246 0.9777 0.9948 1.0000

Just to show that the Eigenvalues sum to the variance in the data consider:

R> sum(pca$sdev^2) ## sum eigenvalues
[1] 4.573

is the same as summing the individual variances of the the variables

R> apply(iris[, -5], 2, var)
Sepal.Length  Sepal.Width Petal.Length  Petal.Width 
      0.6857       0.1900       3.1163       0.5810 
R> sum(apply(iris[, -5], 2, var))
[1] 4.573

This is the total variance, which the PCA has decomposed into the 4 components.

This allows me to translate what Matlab is showing us. The first set of values are the Eigenvectors of the solution. In R these are:

R> pca$rotation
                  PC1      PC2      PC3     PC4
Sepal.Length  0.36139 -0.65659  0.58203  0.3155
Sepal.Width  -0.08452 -0.73016 -0.59791 -0.3197
Petal.Length  0.85667  0.17337 -0.07624 -0.4798
Petal.Width   0.35829  0.07548 -0.54583  0.7537

What Matlab labels as latent are the Eigenvalues, $\lambda_i$ and the ans are these expressed as a cumulative proportion as I showed above.

So the variance explained by each component is:

R> pca$sdev^2 / sum(pca$sdev^2)
[1] 0.924619 0.053066 0.017103 0.005212

One way to decide how many component you would retain an interpret is a screeplot of the Eigenvalues. In R you get this via

screeplot(pca, type = "l")

which looks like this:

enter image description here

The general idea is to retain component up to the elbow in the screeplot. In this case there is really a single strong component and then you might care to retain the second component as after this the 3rd and 4th components aren't really explaining much variation.

Note the above was done on an analysis of the covariance matrix of the data. Where the data are in different units you will want to scale the data and hence do the PCA on the correlation matrix. But here I think this is OK without scaling here.

$\endgroup$
3
  • 1
    $\begingroup$ +1 Nice forensic analysis of what Matlab is doing--it illustrates more generally how one can go about checking statistical output. $\endgroup$
    – whuber
    Jan 9, 2014 at 22:36
  • $\begingroup$ Thank you very much! your reply has really helped me! If I ask you one more thing, will I bother you too much? $\endgroup$
    – user19565
    Jan 11, 2014 at 23:29
  • $\begingroup$ The case is, I need to show in matlab the graph which shows 3 classes of the IRIS data set, as together with PCA analysis, I am confused with biplot because it shows all data in one color. $\endgroup$
    – user19565
    Jan 11, 2014 at 23:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.