6
$\begingroup$

I am confusing about the relationship between z-score and the normal distribution.

Do we apply z-score normalization to get a data with normal distribution, or given we have a normal distribution, we can apply and use z-score?

What is the exact relationship?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ People will often prefer the term "standardization" in that contest, since "normalization" might be understood as transforming the marginal distribution to have a normal distribution, e.g., with the inverse normal transformation $\endgroup$
    – Firebug
    Mar 14 at 14:58
  • 2
    $\begingroup$ @Firebug "normalization" also has two additional (related but not equivalent) meanings of 1) "rescaling the data within a specific range" and 2) "rescaling a vector to a unit vector due to dividing by its norm". $\endgroup$
    – shadowtalker
    Mar 15 at 18:40

3 Answers 3

Reset to default
5
$\begingroup$

The ``normal distribution'' is an entire family of different distributions. We use the notation $\textbf{Normal}(\mu,\sigma^2)$ to indicate what type of normal we get. If you pick a certain choice for $\mu$ and you pick another choice (positive) for $\sigma$, then you get a different type of Normal. Here are some pictures taken from Wikipedia: enter image description here

If you vary $\mu$ you are varying where the center of the distribution. If you vary $\sigma$ you are varying how spread out the distribution is. The "standard normal distribution", is the one where $\mu=0$ and $\sigma = 1$.

Since there is an infinite family of normal distributions it would be annoying to have different functions/calculators for each one. So it is convenient to convert all normal distributions into the "standard normal" form.

If $x_1,x_2,...,x_n$ are samples from some normal distribution you replace each $x_i$ in that list by, $$ x_i \mapsto \frac{x_i - (\text{sample mean})}{(\text{sample deviation})}$$ This is known as "calculating the z-score for each $x_i$". By doing this process you have transformed your original data set $x_1,...,x_n$ into a new data set $z_1,...,z_n$, but in such a way so that the new data set follows a normal distribution of type $\text{Normal}(0,1)$.

Improve this answer
$\endgroup$
3
  • $\begingroup$ thank you for the reply! so I can only apply z-score to a normal distribution, which convert into a standard normal distribution? but can i apply z-score to say a poisson distribution? $\endgroup$
    – Math Avengers
    Mar 14 at 5:52
  • 2
    $\begingroup$ @MathAvengers You can apply the z-score to any distribution, however, the resulting new distribution will not necessarily be normal! In order to preserve normality, the original distribution must be normal itself. The good news is that most distributions in "the real world" are normal, which is why using the z-score is often a safe thing to do. Are you familiar with R software? If you are, I can show you a very easy example how the z-score goes very poorly when you have a Poisson distribution. $\endgroup$
    – Nicolas Bourbaki
    Mar 15 at 1:29
  • 1
    $\begingroup$ I disagree that most distributions in the real world are Normal. But there are several ways to make a distribution look more Normal, and specifically in the case of sample means we can use the CLT and T-to-Normal asymptotic convergence. $\endgroup$
    – shadowtalker
    Mar 15 at 18:39
7
$\begingroup$

There is no relationship. The (sample) z-score is defined as

$$ z_i = \frac{ x_i - \bar x } {s} $$

where $i$ indexes observations $\{x\}$, $\bar x$ is the sample mean, and $s$ is the sample standard deviation.

There is nothing in this definition which states that the data has to be normally distributed, or that you can get normally distributed data by applying this transformation.

The z-score represents the number of standard deviations that a data is from the mean.

You may be getting mixed up with a z-test. In a z-test, we assume that under the null hypothesis, the test statistic of interest (e.g. a sample mean) has a normal distribution. The z-test procedure can be found here.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ I wouldn't say there's no relationship. The generic "z score" appears verbatim as part of the Normal PDF, because the mean and standard deviation are location and scale parameters of the Normal distribution. $\endgroup$
    – shadowtalker
    Mar 15 at 18:44
5
$\begingroup$

I think there is some confusion here due to the word "normalization". In this context, normalization means that the data are transformed to have zero mean and unit standard deviation. The transformed data will also be dimensionless, i.e. lacking physical units.

Z-score normalization does not mean that the data become normally distributed. The transformed data are "normal" only in the sense of having zero mean and unit standard deviation.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.