1
$\begingroup$

I am very confused as I am reading through PCA. Some sources say that I should normalize my data before applying PCA, and some sources say that I should standardize my data before applying PCA. I know that normalization will only change the scale of my values into a range of [0,1]. On the other hand, when standardizing, I am changing variables' means to 0 and standard deviations to 1.

Sources say that I should standardize my variables: https://onlinecourses.science.psu.edu/stat505/node/55

http://sebastianraschka.com/Articles/2014_about_feature_scaling.html#the-effect-of-standardization-on-pca-in-a-pattern-classification-task

Sources say that I should normalize my variables:

https://datafai.com/2017/10/27/data-standardization-or-normalization/

Why do we need to normalize data before principal component analysis (PCA)?

$\endgroup$
  • 3
    $\begingroup$ I've never seen a definition of "normalization" that involves changing the scale of variables onto the range of [0,1]; perhaps that is a terminology that comes up in certain fields, but it is certainly not a widely used definition. And, in fact, the last link you provide (to another CrossValidated question) seems to me to make it clear that they are referring to standardization (e.g. to mean 0 and st.dev. 1). The accepted answer in that thread provides the intuition as to why this is important; that author used the term "normalization" but in the comments clarifies they meant "standardization" $\endgroup$ – Ryan Simmons Apr 24 '18 at 21:27
  • $\begingroup$ Yes, I found that definition of normalization online. I am just trying to see why are people keep changing their terminologies between normalization and standardization. The last link is about standardization, but the questioner asks for normalization. It also seems the best answer has also confused between normalization and standardization. Wikipedia says 'Feature scaling is used to bring all values into the range [0,1]' under the Normalization (statistics) page. $\endgroup$ – lusicat Apr 24 '18 at 21:31
  • $\begingroup$ I don't know who (meaning which individuals) keeps changing their terminologies. What's much more obvious is a culture or group difference: machine learning people are using many terms in senses different from those earlier established in statistics (as well as contributing evocative newer names). There is some terrible terminology in statistics (why do terms such as dependent and independent variables survive?) and naming things after people is double-edged, but normalized is already so overloaded in mathematical sciences that it didn't need another meaning. $\endgroup$ – Nick Cox Apr 24 '18 at 22:09
3
$\begingroup$

The Wikipedia page on "Normalization" notes:

In statistics and applications of statistics, normalization can have a range of meanings.

It then goes on to list 6 examples of "normalizations in statistics," including both of what you have called "standardization" and "normalization."

"Normalization" onto [0,1] is called "feature scaling" or "unity-based normalization" on the Wikipedia page. "Normalization" based on the observed mean and standard deviation (called "Student's t-statistic" on that page; "standardization" in more frequent but not universal usage) is typically what you want for PCA.

This type of terminological confusion happens often in practice. Consider, for example, the frequent use of "multivariate" to represent multiple predictors in a model, when that word might best be reserved for situations with multiple types of outcomes.

So I wouldn't worry too much about the terminology that other people are using. Look into what they actually did, not what they called it. Then when you report your study, explain clearly what you did and try to use the best current terminology yourself.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.