Standard deviation for a half-normal distribution

Question

I have a vector of numbers which is generated from a normal distribution with standard deviation $\sigma$ and $\mu = 0$. When I'm turning these numbers into absolute values, I suppose that the they will follow half-normal distribution with the same standard deviation.

My question is, if I initially start at having a vector of absolute values which generated from half-normal distribution, would it be mathematically sound to add the negative values of these numbers and then find the combined standard deviation between absolute values and negative values to recreate the normal distribution?

Answer 1

1. The half normal distribution is usually parametrised so that the $\sigma$ from the corresponding normal distribution is its scale parameter. However, this is not the standard deviation of it, which is $\sigma\sqrt{1-\frac{2}{\pi}}$, see https://en.wikipedia.org/wiki/Half-normal_distribution. It should be intuitive that the sd is smaller than $\sigma$; there is more variance if observations can deviate on both sides of zero - cutting one half off a region where no observations is left and putting them in the place where the other observations are lowers variation.

2. Regarding your question: If you want a proper random sample from the normal distribution, this is not correct, because although the normal distribution is symmetric, samples of it are not perfectly symmetric. Particularly the probability is zero that if you have $x$ in a normal sample you also have $-x$ in it. The number of observations left and right from 0 can be the same, but more often than not, due to random variation, it won't be the same.

Answer 2

If you randomly assign a sign (+ or -) to each member of the sample generated from a half-normal distribution, the result will be indistinguishable from a sample generated from the corresponding unfolded normal distribution.

Your method wouldn't work - for example, if your sample size is 2, it would be very surprising to draw the sample 4, -4 from a normal distribution with mean zero and variance 1, since it would involve two separate 4-sigma events; but it would be much less surprising to generate it using your method from a single sample from the corresponding half-normal distribution, as that would only involve a single 4-sigma event.

Answer 3

With data $x_1, x_2, \ldots, x_n,$ you propose synthesizing a dataset of $2n$ values $|x_1|, -|x_1|, |x_2|, -|x_2|, \ldots, |x_n|, -|x_n|.$ Because each absolute value $|x_i|$ balances its negative $-|x_i|,$ the mean is zero. The usual standard deviation estimator therefore reduces to the square root of

$$\frac{1}{2n-1}\left((|x_1|-0)^2 + (-|x_1|-0)^2 + \cdots + (-|x_n|-0)^2\right) = \frac{1}{n-1/2}\sum_{i=1}^n x_i^2.$$

As we will see, this needs a slight modification to work well.

---

One way to derive an answer uses the method of maximum likelihood. One definition of the half-normal distribution with standard deviation $\sigma$ is that the probability density of any value $x\ge 0$ is proportional to $\exp(-(x/\sigma)^2/2) / \sigma.$ (Notice how extremely close that is to the definition of a Normal distribution: the only difference is the restriction $x\ge 0.$)

Thus, given a dataset of (absolute) values $\mathbf{x}=x_1, x_2, \ldots, x_n$ drawn independently from such a distribution, the log likelihood takes the form

$$\Lambda(\sigma, \mathbf{x}) = C(n) -\sum_{i=1}^n \left(\log\sigma + \frac{1}{2}\left(\frac{x_i}{\sigma}\right)^2\right)$$

which attains its maximum either as $\sigma\to0,$ as $\sigma\to\infty,$ or where the derivative of $\Lambda$ is zero; that is, at the solutions to

$$0 =\frac{\mathbf{d}}{\mathbf{d}\sigma}\Lambda(\sigma,\mathbf x) = -\frac{n}{\sigma} + \frac{1}{\sigma^3}\sum_{i=1}^n x_i^2.$$

You can check that (unless all the $x_i$ are equal) the values at $0$ and $\infty$ are not solutions. (When all the $x_i$ are equal, the unique maximum occurs as $\hat\sigma\to0.$) The resulting estimate is

$$\hat\sigma^2 = \frac{1}{n}\sum_{i=1}^n x_i^2.$$

The maximum likelihood estimate of $\hat\sigma$ will be the square root of this quantity.

Now suppose, as you propose, we were to replace the data $(x_i)$ with a dataset twice this size by introducing the negatives of the $x_i.$ This has the following obvious effects:

1. The count doubles from $n$ to $2n.$

2. The mean of the new data is $0.$

Therefore the standard deviation of the new data is

$$s^2 = \frac{1}{2n}\left(\sum_{i=1}^n (x_i-0)^2 + \sum_{i=1}^n (-x_i-0)^2\right) = \frac{1}{2n}\sum_{i=1}^n 2x_i^2 = \hat\sigma^2$$

provided we compute it using a denominator of $2n$ rather than a denominator of $2n-1$ as I sketched at the outset. With this small modification, your procedure is identical to the maximum likelihood estimate.

It is worthwhile to inquire what properties this estimator has. The property usually invoked to justify using $n-1$ in the denominator is that $\hat \sigma^2$ be an unbiased estimator of the variance. Let us therefore compute that expectation:

$$E[\hat\sigma^2] = E\left[\frac{1}{n}\sum_{i=1}^n x_i^2\right] = \frac{1}{n}\sum_{i=1}^n E\left[x_i^2\right] = \frac{1}{n}\sum_{i=1}^n \sigma^2 = \sigma^2.$$

$\hat \sigma^2$ is an unbiased estimator of the variance.

This reinforces the growing sense that your estimator is a good one. Moreover, you may now exploit all the additional properties of maximum likelihood estimation to develop confidence intervals for $\sigma,$ etc. Be careful, though, not to make the mistake of using $2n$ for the dataset size! You have only $n$ data values and the uncertainties in your estimates and predictions need to reflect that number rather than being based on the larger $2n.$