This question already has an answer here:

Often when PCA is performed on exam results where all variables (dimensions) have the same $0$ to $100$ scale, scaling is none the less applied. For different scales I can see the purpose of it, but not in this case. Why is it done?


marked as duplicate by amoeba, Sycorax, Community Apr 27 '16 at 8:34

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • $\begingroup$ This is covered in various answers to our very popular thread on this topic: stats.stackexchange.com/questions/53. Look through all well-upvoted answers to get a feeling of different opinions and use cases. $\endgroup$ – amoeba Apr 26 '16 at 15:12
  • $\begingroup$ @amoeba This question may be subtly different from the others: given that exam scores have a natural, common range, why would anyone ever standardize the scores? The accepted (and highly upvoted) answer to the proposed duplicate specifically suggests they would not. Other answers also suggest as much. $\endgroup$ – whuber Apr 26 '16 at 16:59
  • $\begingroup$ @whuber I agree that this question has its own spin (+1, by the way). I think the answers to the proposed duplicate all together (not only the accepted one) do provide some guidance and discussion of both options (standardizing or not). I am happy for this thread to stay open and alone, but I am afraid it will not get as thorough a discussion as already exists elsewhere... $\endgroup$ – amoeba Apr 26 '16 at 19:38
  • $\begingroup$ @whuber but what if there are topics that really 'discriminate' students so that the variance for these particular topics are much greater than the other topics ? It is the same potential range but woudn't you end up with a really distorted space ? Wouldn't scaling be advisable in that situation ? $\endgroup$ – Riff Apr 27 '16 at 6:58
  • $\begingroup$ @Nicolas I'm not sure what you mean by "topic" or "really distorted space." If your point is that different exam variables can have substantially different variances, then that would imply the fact they have natural limits of $0$ and $100$ is irrelevant--and likewise this question becomes irrelevant (or trivial). Many tests, though, are designed so that (at least in some reference population) the variances of each variable are all equal. $\endgroup$ – whuber Apr 27 '16 at 11:33

Scaling (for PCA) is kind of a personal matter. Some people always do it, others won't, whatever the data.

It is not just a question of measuring scale (°C vs °K, km vs miles), not scaling leads to giving more importance to variables that have larger variance (such variables would contribute more to dimensions construction than other variables). Some people exactly want that as they reckon that variables with small variance are of little interest. On the other hand, people that do scale their variables often state that all variables are of equal interest: a variable with small variance may even be more interesting - for exemple in sensometrics, a difficult item (umami for european people) to evaluate and thus be a key item to separate your products while other items (sweet taste) will have larger variance as it is easily recognized and people give notes on a much more individual level-

When you have variables on different scales, for exemple $m$ and $km$, the difference of scale often leads to difference in variance that may not be relevant (just an artificial product of scale) but scaling in a PCA is much more than just getting rid of that problem.

  • $\begingroup$ OK Nicolas, I thought about your points. Whilst I am not a statistician, but did study strd dev and variance at uni many years ago, reading your answer makes me think: $\endgroup$ – thebluephantom Apr 27 '16 at 16:06
  • $\begingroup$ 1) whilst I can see that maths is more difficult than econ imho, one can argue that it depends on the individual's abilities, that is not taken into account I think. So, if I just want to cluster on 0 .. 100 regardless of any interpretation of the exam subject, then reading your response and the fact that I want to consider all results as equal makes me think I would NOT want to scale. But may be I see that entirely wrong. $\endgroup$ – thebluephantom Apr 27 '16 at 16:17
  • $\begingroup$ 2) I am getting the impression that in machine learning all the algorithms need normalization except decision trees, frequent item sets. 3) I have also read that statisticians do not like skewed data. Seems a little akin here. I would think I would like to know if the data is skewed. It's interesting. $\endgroup$ – thebluephantom Apr 27 '16 at 16:17

Not the answer you're looking for? Browse other questions tagged or ask your own question.