5
$\begingroup$

I have a vector of numbers which is generated from a normal distribution with standard deviation $\sigma$ and $\mu = 0$. When I'm turning these numbers into absolute values, I suppose that the they will follow half-normal distribution with the same standard deviation.

My question is, if I initially start at having a vector of absolute values which generated from half-normal distribution, would it be mathematically sound to add the negative values of these numbers and then find the combined standard deviation between absolute values and negative values to recreate the normal distribution?

$\endgroup$

3 Answers 3

6
$\begingroup$
  1. The half normal distribution is usually parametrised so that the $\sigma$ from the corresponding normal distribution is its scale parameter. However, this is not the standard deviation of it, which is $\sigma\sqrt{1-\frac{2}{\pi}}$, see https://en.wikipedia.org/wiki/Half-normal_distribution. It should be intuitive that the sd is smaller than $\sigma$; there is more variance if observations can deviate on both sides of zero - cutting one half off a region where no observations is left and putting them in the place where the other observations are lowers variation.

  2. Regarding your question: If you want a proper random sample from the normal distribution, this is not correct, because although the normal distribution is symmetric, samples of it are not perfectly symmetric. Particularly the probability is zero that if you have $x$ in a normal sample you also have $-x$ in it. The number of observations left and right from 0 can be the same, but more often than not, due to random variation, it won't be the same.

$\endgroup$
5
  • 1
    $\begingroup$ In R, code set.seed(2021)z = rnorm(10^6); sd(abs(z)) returns $0.6039879$ and sqrt(1 - 2/pi) returns $0.6028103.$ With a million iterations, it is reasonable to explect 2 and 2 place accuracy. $\endgroup$
    – BruceET
    Jul 18, 2021 at 21:35
  • 2
    $\begingroup$ This answer relies on confusing the parameter of the Normal distribution with the standard deviation of the corresponding half-Normal distribution. I cannot see that such a confusion is present in the original question. $\endgroup$
    – whuber
    Jul 19, 2021 at 13:33
  • 1
    $\begingroup$ @whuber: The original question says "I suppose that the they will follow half-normal distribution with the same standard deviation". $\endgroup$ Jul 19, 2021 at 14:36
  • $\begingroup$ Yes: but in the context isn't it evident that the term "standard deviation" must be referring to the parameter of the parent Normal distribution and not to a particular function of its moments that happens to go by the same name? $\endgroup$
    – whuber
    Jul 19, 2021 at 14:50
  • 1
    $\begingroup$ @whuber: I'd normally assume that somebody who writes standard deviation means standard deviation. Anyway no harm is done in case the questioner already knew what I explained as item 1. $\endgroup$ Jul 19, 2021 at 15:55
4
$\begingroup$

If you randomly assign a sign (+ or -) to each member of the sample generated from a half-normal distribution, the result will be indistinguishable from a sample generated from the corresponding unfolded normal distribution.

Your method wouldn't work - for example, if your sample size is 2, it would be very surprising to draw the sample 4, -4 from a normal distribution with mean zero and variance 1, since it would involve two separate 4-sigma events; but it would be much less surprising to generate it using your method from a single sample from the corresponding half-normal distribution, as that would only involve a single 4-sigma event.

$\endgroup$
4
  • 1
    $\begingroup$ I don't understand this argument at all. It seems to require that we forget the fact that the $4$ and $-4$ are not independent. Could you provide a rigorous justification for it? $\endgroup$
    – whuber
    Jul 19, 2021 at 13:32
  • $\begingroup$ @whuber I'm saying that drawing a +4 and a -4 (independently, in that order) from a standard normal distribution has a low likelihood (given by $f(4)f(-4) = f(4)^2$ where $f$ is the pdf of the standard normal distribution), but that drawing a single +4 from a standard half-normal distribution and then mirroring it to add a (totally non-independent) -4 to the sample would have a much higher likelihood (just $2f(4)$). Does that clarify? $\endgroup$
    – fblundun
    Jul 19, 2021 at 13:46
  • 1
    $\begingroup$ I just don't see how that's relevant to the question. It does not suppose the duplicated (negated) data are independent of the original data. $\endgroup$
    – whuber
    Jul 19, 2021 at 14:08
  • $\begingroup$ @whuber my understading of the question is that OP hypothesizes that the process for making inferences about the standard deviation of a zero-centered normal distribution based on a sample of size 2n need not take into account whether those data are really independent or are secretly just n pairs of duplicated points from a half-normal distribution. My answer aims to show intuitively that these two algorithms for generating 2n datapoints result in different distributions. $\endgroup$
    – fblundun
    Jul 19, 2021 at 15:45
3
$\begingroup$

With data $x_1, x_2, \ldots, x_n,$ you propose synthesizing a dataset of $2n$ values $|x_1|, -|x_1|, |x_2|, -|x_2|, \ldots, |x_n|, -|x_n|.$ Because each absolute value $|x_i|$ balances its negative $-|x_i|,$ the mean is zero. The usual standard deviation estimator therefore reduces to the square root of

$$\frac{1}{2n-1}\left((|x_1|-0)^2 + (-|x_1|-0)^2 + \cdots + (-|x_n|-0)^2\right) = \frac{1}{n-1/2}\sum_{i=1}^n x_i^2.$$

As we will see, this needs a slight modification to work well.


One way to derive an answer uses the method of maximum likelihood. One definition of the half-normal distribution with standard deviation $\sigma$ is that the probability density of any value $x\ge 0$ is proportional to $\exp(-(x/\sigma)^2/2) / \sigma.$ (Notice how extremely close that is to the definition of a Normal distribution: the only difference is the restriction $x\ge 0.$)

Thus, given a dataset of (absolute) values $\mathbf{x}=x_1, x_2, \ldots, x_n$ drawn independently from such a distribution, the log likelihood takes the form

$$\Lambda(\sigma, \mathbf{x}) = C(n) -\sum_{i=1}^n \left(\log\sigma + \frac{1}{2}\left(\frac{x_i}{\sigma}\right)^2\right)$$

which attains its maximum either as $\sigma\to0,$ as $\sigma\to\infty,$ or where the derivative of $\Lambda$ is zero; that is, at the solutions to

$$0 =\frac{\mathbf{d}}{\mathbf{d}\sigma}\Lambda(\sigma,\mathbf x) = -\frac{n}{\sigma} + \frac{1}{\sigma^3}\sum_{i=1}^n x_i^2.$$

You can check that (unless all the $x_i$ are equal) the values at $0$ and $\infty$ are not solutions. (When all the $x_i$ are equal, the unique maximum occurs as $\hat\sigma\to0.$) The resulting estimate is

$$\hat\sigma^2 = \frac{1}{n}\sum_{i=1}^n x_i^2.$$

The maximum likelihood estimate of $\hat\sigma$ will be the square root of this quantity.

Now suppose, as you propose, we were to replace the data $(x_i)$ with a dataset twice this size by introducing the negatives of the $x_i.$ This has the following obvious effects:

  1. The count doubles from $n$ to $2n.$

  2. The mean of the new data is $0.$

Therefore the standard deviation of the new data is

$$s^2 = \frac{1}{2n}\left(\sum_{i=1}^n (x_i-0)^2 + \sum_{i=1}^n (-x_i-0)^2\right) = \frac{1}{2n}\sum_{i=1}^n 2x_i^2 = \hat\sigma^2$$

provided we compute it using a denominator of $2n$ rather than a denominator of $2n-1$ as I sketched at the outset. With this small modification, your procedure is identical to the maximum likelihood estimate.

It is worthwhile to inquire what properties this estimator has. The property usually invoked to justify using $n-1$ in the denominator is that $\hat \sigma^2$ be an unbiased estimator of the variance. Let us therefore compute that expectation:

$$E[\hat\sigma^2] = E\left[\frac{1}{n}\sum_{i=1}^n x_i^2\right] = \frac{1}{n}\sum_{i=1}^n E\left[x_i^2\right] = \frac{1}{n}\sum_{i=1}^n \sigma^2 = \sigma^2.$$

$\hat \sigma^2$ is an unbiased estimator of the variance.

This reinforces the growing sense that your estimator is a good one. Moreover, you may now exploit all the additional properties of maximum likelihood estimation to develop confidence intervals for $\sigma,$ etc. Be careful, though, not to make the mistake of using $2n$ for the dataset size! You have only $n$ data values and the uncertainties in your estimates and predictions need to reflect that number rather than being based on the larger $2n.$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.