# Not normalizing data before PCA gives better explained variance ratio

I normalized my dataset then ran 3 component PCA to get small explained variance ratios ([0.50, 0.1, 0.05]).

When I didn't normalize but whitened my dataset then ran 3 component PCA, I got high explained variance ratios ([0.86, 0.06,0.01]).

Since I want to retain as much data into 3 components, should I NOT normalize the data? From my understanding we should always normalize before PCA.

By normalizing: setting mean to 0 and having unit variance.

• Although it is unclear what you mean by "normalizing" the data (I know of at least four standard ways to do this in PCA and likely there are more), it sounds like the material at stats.stackexchange.com/questions/53 might be illuminating.
– whuber
Commented Jul 2, 2014 at 21:12
• Hi whubber: I mean normalizing each observation to unit norm
– user46925
Commented Jul 2, 2014 at 21:16
• Thanks. The usual term for that is "standardizing." When you do that you are performing PCA based on correlations: that's why I think the link I provided might answer your question already. However, I see none of the answers there actually explain why or how you will get different results (perhaps because it's complex and the effect of standardization may be difficult to predict).
– whuber
Commented Jul 2, 2014 at 21:25
• Is whitening before PCA typical? What's the goal of doing that? Commented Jul 3, 2014 at 1:20
• If you were working with images, for example, the norm of the images corresponds to the brightness. The high explained variance of the un-normalized data means that a lot of the data can be explained by changes in brightness. If brightness is not important to you, as it often is not in image processing, then you would want to make all the images unit norm first. Even thought the explained variance of your pca components will be lower, it better reflects what you're interested in. Commented Jul 3, 2014 at 14:25

Depends on the goal of your analysis. Some common practices, some of which are mentioned in whuber's link:

1. Standardizing is usually done when the variables on which the PCA is performed are not measured on the same scale. Note that standardizing implies assigning equal importance to all variables.
2. If they are not measured on the same scale and you choose to work on the non standardized variables, it is often the case that each PC is dominated by a single variable and you just get a sort of ordering of the variables by their variance. (One of the loadings of each (early) component will be close to +1 or -1.)
3. The two methods often lead to different results, as you have experienced.

Intuitive example:

Suppose you have two variables: the height of a tree and the girth of the same tree. We will convert the volume to a factor: a tree will be high in volume if its volume is bigger than 20 cubic feet, and low in volume otherwise. We will use the trees dataset which comes preloaded in R.

>data(trees)
>tree.girth<-trees[,1]
>tree.height<-trees[,2]
>tree.vol<-as.factor(ifelse(trees[,3]>20,"high","low"))


Now suppose that the height was actually measured in miles instead of feet.

>tree.height<-tree.height/5280
>tree<-cbind(tree.height,tree.girth)
>
>#do the PCA
>tree.pca<-princomp(tree)
>summary(tree.pca)
Importance of components:
Comp.1       Comp.2
Standard deviation     3.0871086 1.014551e-03
Proportion of Variance 0.9999999 1.080050e-07
Cumulative Proportion  0.9999999 1.000000e+00


The first component explains almost 100% of the variability in the data. The loadings:

> loadings(tree.pca)

Comp.1 Comp.2
tree.height        -1
tree.girth   1


Graphical assessment:

>biplot(tree.pca,xlabs=tree.vol,col=c("grey","red"))


We see that trees high in volume tend to have a high tree girth, but the three height doesn't give any information on tree volume. This is likely wrong and the consequence of the two different unit measures.

We could use the same units, or we could standardize the variables. I expect both will lead to a more balanced picture of the variability. Of course in this case one can argue that the variables should have the same unit but not be standardized, which may be a valid argument, were it not that we are measuring two different things. (When we would be measuring the weight of the tree and the girth of the tree, the scale on which both should be measured is no longer very clear. In this case we have a clear argument to work on the standardized variables.)

>tree.height<-tree.height*5280
>tree<-cbind(tree.height,tree.girth)
>
>#do the PCA
>tree.pca<-princomp(tree)
> summary(tree.pca)
Importance of components:
Comp.1    Comp.2
Standard deviation     6.5088696 2.5407042
Proportion of Variance 0.8677775 0.1322225
Cumulative Proportion  0.8677775 1.0000000

Comp.1 Comp.2
tree.height -0.956  0.293
tree.girth  -0.293 -0.956

>biplot(tree.pca,xlabs=tree.vol,col=c("grey","red"))


We now see that trees which are tall and have a big girth, are high in volume (bottom left corner), compared to low girth and low height for low volume trees (upper right corner). This intuitively makes sense.

• (+1) Very nice answer, actually surpassing all those given in the linked popular thread about PCA on covariance vs. correlation. Commented Jul 3, 2014 at 10:51
• In the case of standardization, the two variables would always be assigned the same weight. Therefore what would be the point of doing PCA? Isn't the purpose of PCA to capture the main variance of X? (given they are measured in the same scale) Thanks!
– Yuan
Commented Feb 8, 2020 at 9:27
• You can still learn about relationships (correlations) between variables when performing a PCA on standardized data. E.g. the cosine of the angle between the variables in the biplot relates to the correlation between these variables. Commented Feb 26, 2020 at 16:48