# PCA on correlation or covariance?

What are the main differences between performing Principal Components Analysis on a correlation and covariance matrix? Do they give the same results?

-
For additional discussion, please visit stats.stackexchange.com/questions/62677/…. –  whuber Jun 27 at 14:56

You tend to use the covariance matrix when the variable scales are similar and the correlation matrix when variables are on different scales. Using the correlation matrix standardises the data.

In general they give different results. Especially when the scales are different.

As example, take a look a look at this R heptathlon data set. Some of the variables have an average value of about 1.8 (the high jump), whereas other variables (200m) are around 20.

library(HSAUR)
# look at heptathlon data
heptathlon

# correlations
round(cor(heptathlon[,-8]),2)   # ignoring "score"
# covariance
round(cov(heptathlon[,-8]),2)

# PCA
# scale=T bases the PCA on the correlation matrix
hep.PC.cor = prcomp(heptathlon[,-8], scale=TRUE)
hep.PC.cov = prcomp(heptathlon[,-8], scale=FALSE)

# PC scores per competitor
hep.scores.cor = predict(hep.PC.cor)
hep.scores.cov = predict(hep.PC.cov)

# Plot of PC1 vs PC2
par(mfrow = c(2, 1))
plot(hep.scores.cov[,1],hep.scores.cov[,2],
xlab="PC 1",ylab="PC 2", pch=NA, main="Covariance")
text(hep.scores.cov[,1],hep.scores.cov[,2],labels=1:25)

plot(hep.scores.cor[,1],hep.scores.cor[,2],
xlab="PC 1",ylab="PC 2", pch=NA, main="Correlation")
text(hep.scores.cor[,1],hep.scores.cor[,2],labels=1:25)


Notice that the outlying individuals (in this data set) are outliers regardless of whether the covariance or correlation matrix is used.

-
"Using the covariance matrix standardises the data" - I think you mean correlation matrix here...cheers –  Neil McGuigan Nov 5 '10 at 19:05
@neil-mcguigan: Opps. Thanks. –  csgillespie Nov 6 '10 at 21:56
What is the situation, if I convert the variables to z-scores first? –  Jirka-x1 May 18 at 16:00

Bernard Flury, in his excellent book introducing multivariate analysis, described this as an anti-property of principal components. It's actually worse than choosing between correlation or covariance. If you changed the units (e.g. US style gallons, inches etc. and EU style litres, centimetres) you will get substantively different projections of the data.

The argument against automatically using correlation matrices is that it is quite a brutal way of standardising your data. The problem with automatically using the covariance matrix, which is very apparent with that heptathalon data, is that the variables with the highest variance will dominate the first principal component (the variance maximising property).

So the "best" method to use is based on a subjective choice, careful thought and some experience.

-

A late reply, but you may find VERY useful handouts on multivariate data analysis "à la française" on the Bioinformatics department of Lyon. These come from the authors of the R ade4 package. It is in french, though.

-