5
$\begingroup$

Is there a specific name for normalizing some data so that it has mean=0 and sd=1?

Or do I just say "data was normalized to have mean=0 and sd=1"?

$\endgroup$
  • 4
    $\begingroup$ maybe standardized ? $\endgroup$ – George Dontas Aug 20 '10 at 8:34
  • $\begingroup$ btw, this is special case of "whitening" which takes data and makes it's empirical covariance matrix identity $\endgroup$ – Yaroslav Bulatov Aug 24 '10 at 19:30
9
$\begingroup$

The quantity $z = \frac{X - \mu}{\sigma}$ is a standard score. So, standardization is a common way to refer to it.

$\endgroup$
4
$\begingroup$

I think it is just called z-score.

[@ttnphns remark: that is correct, however "z-score" also has other meanings in statistics.

z-standardization, z standard value - probably the most widely used terms for the linear transform to mean 0 and sd 1]

$\endgroup$
  • $\begingroup$ no, it is called z-score (small letter), big letter Z-scores refer to mean = 100 and sd = 10. $\endgroup$ – Henrik Aug 20 '10 at 8:46
  • $\begingroup$ @Henrik I have never heard of something like this; nevertheless I've made the change, it seems lowercase z is used more often. $\endgroup$ – user88 Aug 20 '10 at 9:25
  • $\begingroup$ could be that it is a psychology specific distinction. browsing the web i realized that it could even be specific German. At least I only found it in the German wikipedia (not surprisingly i learned it in a German university): de.wikipedia.org/wiki/Normskala $\endgroup$ – Henrik Aug 20 '10 at 9:50
  • 1
    $\begingroup$ Let's not confuse the z-score with the process of standardization. The former is described in ars' answer: it is a statistic. Standardization consists of replacing each of the X[i] by (X[i]-m)/s for further analysis. That's what nico was asking about. $\endgroup$ – whuber Aug 20 '10 at 15:58
  • $\begingroup$ It is called z-standardization and also mean,sd-standardization. z-score is also a right word for it, although "z-score" is used as well in a different context as a synonym of probit. There exist also Z as a test statistic. $\endgroup$ – ttnphns Dec 28 '19 at 7:38
1
$\begingroup$

As ars' answer states, standardization is the transformation that involves:

  • mean-centering, and
  • rescaling to unit variance.

A generalization of standardization is whitening or sphering whereby a set of one or more variables is linearly-transformed (typically after mean-centering) so that the covariance matrix is the identity matrix. A great reference is:

A. Kessy, A. Lewin, and K. Strimmer, “Optimal whitening and decorrelation,” Statistics, no. May, p. 12, 2015.

$\endgroup$
  • 1
    $\begingroup$ Whitening/sphering needs at least two variables. Your answer to the question is just wrong. $\endgroup$ – ttnphns Dec 28 '19 at 7:40
  • $\begingroup$ @ttnphns I don't see why. The definition of whitening means transforming the inputs so that their covariance matrix is identity. That is meaningful for a number of variables $n \ge 1$. When $n=1$, you get the desired transformation in the question. $\endgroup$ – Neil G Dec 28 '19 at 9:50
  • $\begingroup$ @SycoraxsaysReinstateMonica That's true that mean-centering is separate from variance-altering transformations. However, it is pretty clear that whitening is a generalization of standardization. It says so right in the paper I linked. $\endgroup$ – Neil G Dec 28 '19 at 22:08
  • 1
    $\begingroup$ @SycoraxsaysReinstateMonica I wouldn't say they're conflated distinct concepts. I've edited to explain that one is a generalization of the other. $\endgroup$ – Neil G Dec 28 '19 at 22:15
  • $\begingroup$ The edit looks good! +1 $\endgroup$ – Sycorax says Reinstate Monica Dec 28 '19 at 22:17
0
$\begingroup$

You have to say it was "mean-zero standardized." Dividing by the original calculated standard deviation, $\sigma$, results in a standardized variate with a standard deviation of 1. It has mean zero since it was "centered" (the average was subtracted as well). It's also not normalizing since normalizing implies the final range of the feature (variable) is [0,1], whereas the range of a mean-zero standardized feature is determined from the calculated min and max values, i.e., range= [$Z_{\rm min}, Z_{\rm max}$]. You can also calculate percentiles, which are a type of normalizing, since the final range is [0,1].

Something to note is that mean-zero standardizing and normalizing don't remove skewness, so if a histogram of the original feature (before transformation) had a heavy left or right tail, the tail(s) will still remain in the transformed feature. To get rid of tails after transforming, you have to use the van der Waerden score (VDW).

The van der Waerden (VDW) score for a single observation is merely the inverse cumulative (standard) normal mapping of the observation's percentile value. For example, say you have $n=100$ observations for a continuous variable, you can determine the VDW scores using:

  1. First, sort the values in ascending order, then assign ranks, so you would obtain ranks of $R_i=1,2,\ldots,100.$
  2. Next, determine the percentile for each observations as $pct_i=R_i/(n+1)$.
  3. Once the percentile values are obtained, input them into the inverse mapping function for the CDF of the standard normal distribution, i.e., $N(0,1)$, to obtain the $Z$-score for each, using $Z_i=\Phi^{-1}(pct_i)$.

For example, if you plug in a $pct_i$ value 0.025, you will get $-1.96=\Phi^{-1}(0.025)$. Same goes for a plugin value of $pct_i=0.975$, you'll get $1.96=\Phi^{-1}(0.975)$.

Use of VDW scores is very popular in genetics, where many variables are transformed into VDW scores, and then input into analyses. The advantage of using VDW scores is that skewness and outlier effects are removed from the data, and can be used if the goal is to perform an analysis under the contraints of normality -- and every variable needs to be purely standard normal distributed with no skewness or outliers.

Below is a plot of a log-normal variable V1 based on parameters ln($\mu$) =6, and GSD of 0.3, with its corresponding transforms:

  • n_V1(6,0.3)_1 - normalized into range [0,1]
  • z_V1(6,0.3)_2 - mean-zero standardized into range [$Z_{min}, Z_{max}$]
  • p_V1(6,0.3)_3 - percentiles with range [0,1]
  • vdw_V1(6,0.3)_4 - van der Waerden scores, with usual range [-3,3]

As you can see, the skewness (right tail) is removed by only the VDW transform.

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.