# Unbiased estimator of $\sigma^4$

In the post [here], the user asked the question

$$\{X_i\}_1^n$$ is random sample from $$N(\mu, \sigma^2)$$ with unknown parameters. Find an unbiased estimator of $$\sigma^4$$.

The solution uses a property of Normal Distribution, that of the fourth central moment is proportional to $$\sigma^4$$

$$\operatorname{E}[{(X-\mu)}^4] = 3 \sigma^4$$

to derive

$$\theta_1 = \frac{n-1}{n+1} s^4 = \frac{1}{n^2-1} \left(\sum_{i=1}^n \left(X_i - \overline X\right)^2\right)^2$$

where $$s^2 = \frac{1}{n-1}\sum_{i=1}^n \left(X_i - \bar{X}\right)^2$$, with $$\mathbb E\left[ \theta_1 \right] = \sigma^4$$. This can be re-expressed in terms of power sums, $$S_k= \sum_{i=1}^n X_i^k$$ as

$$\theta_1 = \frac{{\left(n S_2 - S_1^2\right)}^2}{(n^2-1)n}$$

This is also seen in multiple reference books as well.

Comparing to h-statistics, unbiased estimators of central moments, [ref 1], $$\theta_1$$ is also equal to

$$\theta_1 = \left( \frac{n-1}{n+1} \right) h_2^2$$

However, as the Normal Distribution has the property $$\operatorname{E}[{(X-\mu)}^4] = 3 \sigma^4$$, we can also take one-third of $$h_4$$ as an estimator

$$\theta_2 = \left( \frac{1}{3} \right) h_4$$

which according to ref 1 should also be minimum variance unbiased estimator. Although $$\theta_2$$ is an estimator for $$\sigma^4$$ for Normal Distribution, due to property of central moment, it is not the general estimator of $$\sigma^4$$ for any distribution, as $$\sigma^4 = {\left( \sigma_2 \right)}^2 = \sigma_2\, \sigma_2$$ , the product of two central moments. This is the polyache statistic [ref 2], and in particular

$$\theta_3 = h_{2,2}$$

Formula for $$h_{2,2}$$ given in ref 2. (Although not stated, do polyaches also have the minimum variance property as h-statistics?)

While it is true that $$\operatorname{E}(h_2) = \sigma_2$$ that does not imply that $$\operatorname{E}(h_2^2) = {\left( \sigma_2 \right)}^2$$. The unbiased estimator to $$\sigma^4 = {\left( \sigma_2 \right)}^2$$ is the polyache $$h_{2,2}$$

Observation:

(From simulation)

$$\operatorname{Var}(\theta_1) < \operatorname{Var}(\theta_3) < \operatorname{Var}(\theta_2)$$

This I found interesting, especially $$\theta_2$$ as the fourth central moment is just proportional to $$\sigma^4$$ and $$h_4$$ is the minimum variance unbiased estimator to the fourth central moment; yet had the highest variance.

Q1 : Is the order because $$\theta_1$$ was specific to Normal Distribution whereas $$\theta_2$$ and $$\theta_3$$ were applicable to any distribution?

Q2 : $$\sigma^4$$ for the Normal Distribution can be expressed in at least two different ways: First is one-third of fourth central moment; second is square of the variance. The first is specific to the distribution whereas the latter is more general, but both algebraically equal to $$\sigma^4$$; how do we know a priori which estimator to use for minimum variance? Both use direct definitions of h-statistic and polyache.

As a side note, interestingly,

$$n\,(n+1)\,\theta_1 = [ 3(n-1) ] \, \theta_2 + [3+(n-2)n] \, \theta_3$$

EDIT:

Thank you to the comments, they reminded me of Lehmann–Scheffé theorem which explains why $$\theta_1$$ performs the best.

I guess now the question becomes why $$\operatorname{Var}(\theta_1) < \operatorname{Var}(\theta_2)$$, and not equality?

My thinking: $$\theta_2$$ was derived from the fourth central moment, which should not - in general - equal $$\sigma^4$$, but does for the Normal Distribution up to a constant of proportionality. The h-statistics are minimum variance. (ref 1). We used a minimum variance unbiased estimator to estimate $$\sigma^4$$ from an identity of the Normal Distribution, yet in simulation it had the worst variance.

$$\theta_3$$ is most general, as uses the definition of $$\sigma^4$$ directly; yet the variance is in the middle. I suspected that should be worse variance, as is the most general and no Normal Distribution properties or identities were used or implied.

• You should have $\theta_1$ as the MVUE because of en.wikipedia.org/wiki/Lehmann%E2%80%93Scheff%C3%A9_theorem . Nov 30, 2023 at 15:59
• ... and that makes it specific to the sufficient statistics of a normal distribution Nov 30, 2023 at 16:10
• Ah, yes, forgot about Lehmann-Scheff theorem. That does explain $\theta_1$ and its properties. Thank you. Now just confused about $\theta_2$ and $\theta_3$; both relate directly to $\sigma^4$ in two different ways. Nov 30, 2023 at 17:08
• it's not hard to show that $\operatorname{E}(h_2^2) \geq {\left( \sigma_2 \right)}^2$ -- note that $\text{Var}(h_2) = E(h_2^{2})-E(h_2)^{2} \geq 0$, now sub in that $E(h_2)=\sigma_2$ and add $\sigma_2^2$ to both sides. Note that they're only equal when $\text{Var}(h_2)=0$ (consider: when does that happen?). Outside that, positively biased Dec 1, 2023 at 1:41

For the normal distribution we should have $$\theta_1$$ as the MVUE because of https://en.wikipedia.org/wiki/Lehmann%E2%80%93Scheff%C3%A9_theorem
But note that $$\theta_1 =$$ is not in general an unbiased estimator of the fourth moment. It is for a normal distribution, but not in general. So that may be why it is not compared with $$\theta_2$$ when it is stated to be the minimum variance unbiased estimator.