Does standardization compromises Principal Component Analysis? [duplicate]

Question

This question already has an answer here:

• PCA on correlation or covariance? 7 answers

I've been dealing with PCA over the past week and I ended up with a requirement for performing PCA (in Python Machine Learning Book by Sebastian Raschka):

Note that the PCA directions are highly sensitive to data scaling, and we need to standardize the features prior to PCA if the features were measured on different scales and we want to assign equal importance to all features.

However, PCA can be defined as (in Python Machine Learning Book by Sebastian Raschka):

In a nutshell, PCA aims to find the directions of maximum variance in high-dimensional data and projects it onto a new subspace with equal or fewer dimensions that the original one.

So, if we standardize (features rescaled so that they’ll have the properties of a standard normal distribution with μ=0 and σ=1) aren't we eliminating this variance characteristic and ending up with our features being the same?

Answer 1

Yes, feature standardization changes the result of PCA.

Yet if you do not standardize, the feature with the highest variance will dominate your dataset. The absolute variance depends on scaling or units of measurements.

Therefore standardization is allowing us to compare feature relevance independent of scale (in case of z-standardization, where the resulting variance is 1 and the mean is 0).

Further details are to be found in these questions and answers:

Why do we need to normalize data before analysis

PCA on correlation or covariance?

Answer 2

@Nikolas is correct, but I should address your question more directly:

aren't we eliminating this variance characteristic

Yes, the point is to eliminate the variance. We want to eliminate the variability because it's not needed and it influences our PCA analysis.

Let me give you an example. If you have 100 USD and it's equal to 1000 Japanese JEN. Even the money is the same, but if you do nothing our PCA might put more weights on the Japanese JEN... We don't want the variance so we need to standardize it (by converting Japanese to USD).