# Variable standardization / scaling for PCA when all dimensions already have same scale [duplicate]

Often when PCA is performed on exam results where all variables (dimensions) have the same $0$ to $100$ scale, scaling is none the less applied. For different scales I can see the purpose of it, but not in this case. Why is it done?

• This is covered in various answers to our very popular thread on this topic: stats.stackexchange.com/questions/53. Look through all well-upvoted answers to get a feeling of different opinions and use cases. Apr 26, 2016 at 15:12
• @amoeba This question may be subtly different from the others: given that exam scores have a natural, common range, why would anyone ever standardize the scores? The accepted (and highly upvoted) answer to the proposed duplicate specifically suggests they would not. Other answers also suggest as much.
– whuber
Apr 26, 2016 at 16:59
• @whuber I agree that this question has its own spin (+1, by the way). I think the answers to the proposed duplicate all together (not only the accepted one) do provide some guidance and discussion of both options (standardizing or not). I am happy for this thread to stay open and alone, but I am afraid it will not get as thorough a discussion as already exists elsewhere... Apr 26, 2016 at 19:38
• @whuber but what if there are topics that really 'discriminate' students so that the variance for these particular topics are much greater than the other topics ? It is the same potential range but woudn't you end up with a really distorted space ? Wouldn't scaling be advisable in that situation ?
– Riff
Apr 27, 2016 at 6:58
• @Nicolas I'm not sure what you mean by "topic" or "really distorted space." If your point is that different exam variables can have substantially different variances, then that would imply the fact they have natural limits of $0$ and $100$ is irrelevant--and likewise this question becomes irrelevant (or trivial). Many tests, though, are designed so that (at least in some reference population) the variances of each variable are all equal.
– whuber
Apr 27, 2016 at 11:33

Scaling (for PCA) is kind of a personal matter. Some people always do it, others won't, whatever the data.

It is not just a question of measuring scale (°C vs °K, km vs miles), not scaling leads to giving more importance to variables that have larger variance (such variables would contribute more to dimensions construction than other variables). Some people exactly want that as they reckon that variables with small variance are of little interest. On the other hand, people that do scale their variables often state that all variables are of equal interest: a variable with small variance may even be more interesting - for exemple in sensometrics, a difficult item (umami for european people) to evaluate and thus be a key item to separate your products while other items (sweet taste) will have larger variance as it is easily recognized and people give notes on a much more individual level-

When you have variables on different scales, for exemple $m$ and $km$, the difference of scale often leads to difference in variance that may not be relevant (just an artificial product of scale) but scaling in a PCA is much more than just getting rid of that problem.

• OK Nicolas, I thought about your points. Whilst I am not a statistician, but did study strd dev and variance at uni many years ago, reading your answer makes me think: Apr 27, 2016 at 16:06
• 1) whilst I can see that maths is more difficult than econ imho, one can argue that it depends on the individual's abilities, that is not taken into account I think. So, if I just want to cluster on 0 .. 100 regardless of any interpretation of the exam subject, then reading your response and the fact that I want to consider all results as equal makes me think I would NOT want to scale. But may be I see that entirely wrong. Apr 27, 2016 at 16:17
• 2) I am getting the impression that in machine learning all the algorithms need normalization except decision trees, frequent item sets. 3) I have also read that statisticians do not like skewed data. Seems a little akin here. I would think I would like to know if the data is skewed. It's interesting. Apr 27, 2016 at 16:17