# What's the difference between Normalization and Standardization?

At work we were discussing this as my boss has never heard of normalization. In Linear Algebra, Normalization seems to refer to the dividing of a vector by its length. And in statistics, Standardization seems to refer to the subtraction of a mean then dividing by its SD. But they seem interchangeable with other possibilities as well.

When creating some kind of universal score, that makes up 2 different metrics, which have different means and different SD's, would you Normalize, Standardize, or something else? One person told me it's just a matter of taking each metric and dividing them by their SD, individually. Then summing the two. And that will result in a universal score that can be used to judge both metrics.

For instance, say you had the number of people who take the subway to work (in NYC) and the number of people who drove to work (in NYC).

Train ---> x Car ---> y

If you wanted to create a universal score to quickly report traffic fluctuations, you can't just add mean(x) and mean(y) because there will be a LOT more people who ride the train. There's 8 million people living in NYC, plus tourists. That's millions of people taking the train everyday verse hundreds of thousands of people in cars. So they need to be transformed to a similar scale in order to be compared.

If mean(x) = 8,000,000

and mean (y) = 800,000

Would you normalize x & y then sum? Would you standardize x & y then sum? Or would you divide each by their respective SD then sum? In order to get to a number that when fluctuates, represents total traffic fluctuations.

Any article or chapters of books for reference would be much appreciated. THANKS!

Also here's another example of what I'm trying to do.

Imagine you're a college dean, and you're discussing admission requirements. You may want students with at least a certain GPA and a certain test score. It'd be nice if they were both on the same scale because then you could just add the two together and say, "anyone with at least a 7.0 can get admitted." That way, if a prospective student has a 4.0 GPA, they could get as low as a 3.0 test score and still get admitted. Inversely, if someone had a 3.0 GPA, they could still get admitted with a 4.0 test score.

But it's not like that. The ACT is on a 36 point scale and most GPA's are on 4.0 (some are 4.3, yes annoying). Since I can't just add an ACT and GPA to get some kind of universal score, how can I transform them so they can be added, thus creating a universal admission score. And then as a Dean, I could just automatically accept anyone with a score above a certain threshold. Or even automatically accept everyone whose score is within the top 95%.... those sorts of things.

Would that be normalization? standardization? or just dividing each by their SD then summing?

-
The last part of the question sounds like you are trying to create a valuation out of multiple attributes. For more on that see the question and replies at stats.stackexchange.com/q/9137 and stats.stackexchange.com/q/9358 . In particular, note that neither normalization nor standardization have any direct relevance to the Dean's problem. –  whuber May 3 '11 at 22:35

In the business world, "normalization" typically means that the range of values are "normalized to be from 0.0 to 1.0". "Standardization" typically means that the range of values are "standardized" to measure how many standard deviations the value is from its mean. However, not everyone would agree with that. It's best to explain your definitions before you use them.

In any case, your transformation needs to provide something useful.

In your train/car example, do you gain anything out of knowing how many standard deviations from their mean, each value lies? If you plot those "standardized" measures against each other as an x-y plot, you might see a correlation (see the first graph on the right):

http://en.wikipedia.org/wiki/Correlation_and_dependence

If so, does that mean anything to you?

As far as your second example goes, if you want to "equate" a GPA from one scale to another scale, what do these scales have in common? In other words, how could you transform those minimums to be equivalent, and the maximums to be equivalent?

Here's an example of "normalization":

Once you get your GPA and ACT scores in an interchangeable form, does it make sense to weigh the ACT and GPA scores differently? If so, what weighting means something to you?

Edit 1 (05/03/2011) ==========================================

First, I would check out the links suggested by whuber above. The bottom line is, in both of your two-variable problems, you are going to have to come up with an "equivalence" of one variable versus the other. And, a way to differentiate one variable from the other. In other words, even if you can simplify this to a simple linear relationship, you'll need "weights" to differentiate one variable from the other.

Here's an example of a two variable problem:

Multi-Attribute Utilities

From the last page, if you can say that standardized train traffic U1(x) versus standardized car traffic U2(y) is "additively independent", then you might be able to get away with a simple equation such as:

U(x, y) = k1*U1(x) + (1 - k1)*U2(y)


Where k1=0.5 means you're indifferent to standardized car/train traffic. A higher k1 would mean train traffic U1(x) is more important.

However, if these two variables are not "additively independent", then you'll have to use a more complicated equation. One possibility is shown on page 1:

U(x, y) = k1*U1(x) + k2*U2(y) + (1-k1-k2)*U1(x)*U2(y)


In either case, you'll have to come up with a utility U(x, y) that makes sense.

The same general weighting/comparison concepts hold for your GPA/ACT problem. Even if they are "normalized" rather than "standardized".

One last issue. I know you're not going to like this, but the definition of the term "additively independent" is on page 4 of the following link. I looked for a less geeky definition, but I couldn't find one. You might look around to find something better.

Intuitively, the agent prefers being both healthy and wealthy
more than might be suggested by considering the two attributes
separately. It thus displays a preference for probability
distributions in which health and wealth are positively
correlated.


As suggested at the top of this response, if you plot standardized train traffic versus standardized car traffic on an x-y plot, you might see a correlation. If so, then you're stuck with the above non-linear utility equation or something similar.

-
Ok. You're right. It is best to explain my definitions. And in thinking about it again, it's not the definitions that I need. What I need is the appropriate method for creating 1 universal score. Whether that be an Admission score or Traffic score. How does one go about creating a universal metric that's a function of other variables, which were transformed to put them both on a similar scale? And don't worry about the weights. I understand that even just straight summing is weighting the metrics 1/1. But that's less of a concern for me right now. –  Chris May 3 '11 at 21:39
@Chris, I added my answer as an edit above. –  bill_080 May 4 '11 at 3:31
(+1) Good edit. @Chris: you might be interested in the notes to a short set of PowerPoint slides here: this is a presentation on the subject I gave to non-technical people. I mention it because it has some illustrations and guidance for how to "create a universal metric." –  whuber May 4 '11 at 3:41

Normalization rescales the values from to a range of [0,1]. This might useful in some cases where all parameters need to have the same positive scale, but outliers from data set are lost.

Xchanged = (X - Xmin)/(Xmax-Xmin)

Standardization rescales data to have a mean of 0 and standard deviation of 1 (unit variance).

Xchanged = x-mean/sd

For most applications standardization is recommended.

-