6
$\begingroup$

I am very confused as I am reading through PCA. Some sources say that I should normalize my data before applying PCA, and some sources say that I should standardize my data before applying PCA. I know that normalization will only change the scale of my values into a range of [0,1]. On the other hand, when standardizing, I am changing variables' means to 0 and standard deviations to 1.

Sources say that I should standardize my variables: https://onlinecourses.science.psu.edu/stat505/node/55

http://sebastianraschka.com/Articles/2014_about_feature_scaling.html#the-effect-of-standardization-on-pca-in-a-pattern-classification-task

Sources say that I should normalize my variables:

https://datafai.com/2017/10/27/data-standardization-or-normalization/

Why do we need to normalize data before principal component analysis (PCA)?

Improve this question
$\endgroup$
3
  • 3
    $\begingroup$ I've never seen a definition of "normalization" that involves changing the scale of variables onto the range of [0,1]; perhaps that is a terminology that comes up in certain fields, but it is certainly not a widely used definition. And, in fact, the last link you provide (to another CrossValidated question) seems to me to make it clear that they are referring to standardization (e.g. to mean 0 and st.dev. 1). The accepted answer in that thread provides the intuition as to why this is important; that author used the term "normalization" but in the comments clarifies they meant "standardization" $\endgroup$
    – Ryan Simmons
    Commented Apr 24, 2018 at 21:27
  • $\begingroup$ Yes, I found that definition of normalization online. I am just trying to see why are people keep changing their terminologies between normalization and standardization. The last link is about standardization, but the questioner asks for normalization. It also seems the best answer has also confused between normalization and standardization. Wikipedia says 'Feature scaling is used to bring all values into the range [0,1]' under the Normalization (statistics) page. $\endgroup$
    – lusicat
    Commented Apr 24, 2018 at 21:31
  • 1
    $\begingroup$ I don't know who (meaning which individuals) keeps changing their terminologies. What's much more obvious is a culture or group difference: machine learning people are using many terms in senses different from those earlier established in statistics (as well as contributing evocative newer names). There is some terrible terminology in statistics (why do terms such as dependent and independent variables survive?) and naming things after people is double-edged, but normalized is already so overloaded in mathematical sciences that it didn't need another meaning. $\endgroup$
    – Nick Cox
    Commented Apr 24, 2018 at 22:09

1 Answer 1

Reset to default
4
$\begingroup$

The Wikipedia page on "Normalization" notes:

In statistics and applications of statistics, normalization can have a range of meanings.

It then goes on to list 6 examples of "normalizations in statistics," including both of what you have called "standardization" and "normalization."

"Normalization" onto [0,1] is called "feature scaling" or "unity-based normalization" on the Wikipedia page. "Normalization" based on the observed mean and standard deviation (called "Student's t-statistic" on that page; "standardization" in more frequent but not universal usage) is typically what you want for PCA.

This type of terminological confusion happens often in practice. Consider, for example, the frequent use of "multivariate" to represent multiple predictors in a model, when that word might best be reserved for situations with multiple types of outcomes.

So I wouldn't worry too much about the terminology that other people are using. Look into what they actually did, not what they called it. Then when you report your study, explain clearly what you did and try to use the best current terminology yourself.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ +1. In the context of PCA, scaling to [0,1] is a weird choice because this will fail to center the variables. OP can look here for possible consequences: stats.stackexchange.com/questions/22329. $\endgroup$
    – amoeba
    Commented Apr 24, 2018 at 21:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.