# PCA on correlation or covariance?

What are the main differences between performing Principal Components Analysis on a correlation and covariance matrix? Do they give the same results?

-
For additional discussion, please visit stats.stackexchange.com/questions/62677/…. –  whuber Jun 27 at 14:56

You tend to use the covariance matrix when the variable scales are similar and the correlation matrix when variables are on different scales. Using the correlation matrix standardises the data.

In general they give different results. Especially when the scales are different.

As example, take a look a look at this R heptathlon data set. Some of the variables have an average value of about 1.8 (the high jump), whereas other variables (200m) are around 20.

library(HSAUR)
# look at heptathlon data
heptathlon

# correlations
round(cor(heptathlon[,-8]),2)   # ignoring "score"
# covariance
round(cov(heptathlon[,-8]),2)

# PCA
# scale=T bases the PCA on the correlation matrix
hep.PC.cor = prcomp(heptathlon[,-8], scale=TRUE)
hep.PC.cov = prcomp(heptathlon[,-8], scale=FALSE)

# PC scores per competitor
hep.scores.cor = predict(hep.PC.cor)
hep.scores.cov = predict(hep.PC.cov)

# Plot of PC1 vs PC2
par(mfrow = c(2, 1))
plot(hep.scores.cov[,1],hep.scores.cov[,2],
xlab="PC 1",ylab="PC 2", pch=NA, main="Covariance")
text(hep.scores.cov[,1],hep.scores.cov[,2],labels=1:25)

plot(hep.scores.cor[,1],hep.scores.cor[,2],
xlab="PC 1",ylab="PC 2", pch=NA, main="Correlation")
text(hep.scores.cor[,1],hep.scores.cor[,2],labels=1:25)


Notice that the outlying individuals (in this data set) are outliers regardless of whether the covariance or correlation matrix is used.

-
"Using the covariance matrix standardises the data" - I think you mean correlation matrix here...cheers –  Neil McGuigan Nov 5 '10 at 19:05
@neil-mcguigan: Opps. Thanks. –  csgillespie Nov 6 '10 at 21:56
What is the situation, if I convert the variables to z-scores first? –  Jirka-x1 May 18 at 16:00

The arguments based on scale (for variables expressed in the same physical units) seem rather weak. Imagine a set of (dimensionless) variables whose standard deviations vary between 0.001 and 0.1. Compared to a standardized value of 1, these both seem to be 'small' and comparable levels of fluctuations. However, when you express them in decibel, this gives a range of -60 dB against -10 and 0 dB, respectively. Then this would probably then be classified as a 'large range' -- especially if you would include a standard deviation close to 0, i.e., minus infinity dB.

My suggestion would be to do BOTH a correlation- and covariance-based PCA. If the two give the same (or very similar, whatever this may mean) PCs, then you can be reassured you've got an answer that is meaningul. If they give widely different PCs don't use PCA, because two different answers to one problem is not sensible way to solve questions.

-
(-1) Getting "two different answers to the same problem" often just means you're bashing away mindlessly without thinking about which technique is appropriate for your analytical aims. It does not mean that one or (as you state) both techniques are not sensible, but only that at least one might not be appropriate for the problem or the data. Furthermore, in many cases you can anticipate that covariance-based PCA and correlation-based PCA should give different answers. After all, they are measuring different aspects of the data. Doing both by default would not make sense. –  whuber Jun 27 at 15:00

If you have variables with widely varying scales, that is, caloric intake per day, gene expression, ELISA/Luminex in units of ug/dl, ng/dl, based on several orders of magnitude of protein expression, then use correlation as an input to PCA. However, if all of your data are based on e.g. gene expression from the same platform with similar range and scale, or you are working with log equity asset returns, then using correlation will throw out a tremendous amount of information. You actually don't need to think about the difference of using the correlation matrix R or covariance matrix C as an input to PCA, but rather, look at the diagonal values of C and R. You may observe a variance of 100 for one variable, and 10 on another -- which are on the diagonal of C. But when looking at the correlations, the diagonal contains all ones, so the variance of each variable is essentially changed to one as you use the R matrix. In summary, use the correlation matrix R when within-variable range and scale widely differs, and use C to preserve variance if the range and scale of variables is similar or in the same units of measure.

-

A late reply, but you may find VERY useful handouts on multivariate data analysis "à la française" on the Bioinformatics department of Lyon. These come from the authors of the R ade4 package. It is in french, though.

-