2
$\begingroup$

Regarding the differences between "Normalization" and "Standardization," I found that:

Normalization: Is the process of making a dataset having a specified range, probably [0,1] following this equation: $$ x = {x - \min(x) \over \max(x)-\min(x)} $$

Standardization: Refers to making the dataset/ layer have Zero-mean and Unit-variance $$ x = {x - \mu_x \over \sigma_x} $$

However, during BatchNorm, we do standardization not Normalization. Am I correct?

$\endgroup$
1
  • $\begingroup$ Welcome to the "naming" problem. Arguably the hardest problem in the sciences (also in programming). People choose a name (or notation) they think sounds "cool" or "fine" to allow them to get a paper out quickly, without too much thought if its semantics are adequately captured by that name (if at all possible). Then people start using it, and that's that. That's how we end up with terms like "ChatGPT hallucinates" instead of "confabulates", for instance. $\endgroup$ Commented Jul 1, 2023 at 9:06

1 Answer 1

5
$\begingroup$

Those names are not standardized (pun intended), "normalization", "standardization", and "scaling" are often used interchangeably for different transformations.

See for example this Wikipedia article or this one that says

This process of converting a raw score into a standard score is called standardizing or normalizing (however, "normalizing" can refer to many types of ratios; see normalization for more).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.