4
$\begingroup$

In the post [here], the user asked the question

$\{X_i\}_1^n$ is random sample from $N(\mu, \sigma^2)$ with unknown parameters. Find an unbiased estimator of $\sigma^4$.

The solution uses a property of Normal Distribution, that of the fourth central moment is proportional to $\sigma^4$

$$\operatorname{E}[{(X-\mu)}^4] = 3 \sigma^4$$

to derive

$$\theta_1 = \frac{n-1}{n+1} s^4 = \frac{1}{n^2-1} \left(\sum_{i=1}^n \left(X_i - \overline X\right)^2\right)^2$$

where $s^2 = \frac{1}{n-1}\sum_{i=1}^n \left(X_i - \bar{X}\right)^2 $, with $\mathbb E\left[ \theta_1 \right] = \sigma^4$. This can be re-expressed in terms of power sums, $S_k= \sum_{i=1}^n X_i^k$ as

$$\theta_1 = \frac{{\left(n S_2 - S_1^2\right)}^2}{(n^2-1)n}$$

This is also seen in multiple reference books as well.

Comparing to h-statistics, unbiased estimators of central moments, [ref 1], $\theta_1$ is also equal to

$$\theta_1 = \left( \frac{n-1}{n+1} \right) h_2^2 $$

However, as the Normal Distribution has the property $\operatorname{E}[{(X-\mu)}^4] = 3 \sigma^4$, we can also take one-third of $h_4$ as an estimator

$$\theta_2 = \left( \frac{1}{3} \right) h_4 $$

which according to ref 1 should also be minimum variance unbiased estimator. Although $\theta_2$ is an estimator for $\sigma^4$ for Normal Distribution, due to property of central moment, it is not the general estimator of $\sigma^4$ for any distribution, as $\sigma^4 = {\left( \sigma_2 \right)}^2 = \sigma_2\, \sigma_2$ , the product of two central moments. This is the polyache statistic [ref 2], and in particular

$$\theta_3 = h_{2,2} $$

Formula for $h_{2,2}$ given in ref 2. (Although not stated, do polyaches also have the minimum variance property as h-statistics?)

While it is true that $\operatorname{E}(h_2) = \sigma_2$ that does not imply that $\operatorname{E}(h_2^2) = {\left( \sigma_2 \right)}^2$. The unbiased estimator to $\sigma^4 = {\left( \sigma_2 \right)}^2$ is the polyache $h_{2,2}$

Observation:

(From simulation)

$$\operatorname{Var}(\theta_1) < \operatorname{Var}(\theta_3) < \operatorname{Var}(\theta_2) $$

This I found interesting, especially $\theta_2$ as the fourth central moment is just proportional to $\sigma^4$ and $h_4$ is the minimum variance unbiased estimator to the fourth central moment; yet had the highest variance.

Q1 : Is the order because $\theta_1$ was specific to Normal Distribution whereas $\theta_2$ and $\theta_3$ were applicable to any distribution?

Q2 : $\sigma^4$ for the Normal Distribution can be expressed in at least two different ways: First is one-third of fourth central moment; second is square of the variance. The first is specific to the distribution whereas the latter is more general, but both algebraically equal to $\sigma^4$; how do we know a priori which estimator to use for minimum variance? Both use direct definitions of h-statistic and polyache.

As a side note, interestingly,

$$n\,(n+1)\,\theta_1 = [ 3(n-1) ] \, \theta_2 + [3+(n-2)n] \, \theta_3 $$

EDIT:

Thank you to the comments, they reminded me of Lehmann–Scheffé theorem which explains why $\theta_1$ performs the best.

I guess now the question becomes why $\operatorname{Var}(\theta_1) < \operatorname{Var}(\theta_2) $, and not equality?

My thinking: $\theta_2$ was derived from the fourth central moment, which should not - in general - equal $\sigma^4$, but does for the Normal Distribution up to a constant of proportionality. The h-statistics are minimum variance. (ref 1). We used a minimum variance unbiased estimator to estimate $\sigma^4$ from an identity of the Normal Distribution, yet in simulation it had the worst variance.

$\theta_3$ is most general, as uses the definition of $\sigma^4$ directly; yet the variance is in the middle. I suspected that should be worse variance, as is the most general and no Normal Distribution properties or identities were used or implied.

$\endgroup$
4
  • $\begingroup$ You should have $\theta_1$ as the MVUE because of en.wikipedia.org/wiki/Lehmann%E2%80%93Scheff%C3%A9_theorem . $\endgroup$ Nov 30, 2023 at 15:59
  • $\begingroup$ ... and that makes it specific to the sufficient statistics of a normal distribution $\endgroup$
    – Henry
    Nov 30, 2023 at 16:10
  • $\begingroup$ Ah, yes, forgot about Lehmann-Scheff theorem. That does explain $\theta_1$ and its properties. Thank you. Now just confused about $\theta_2$ and $\theta_3$; both relate directly to $\sigma^4$ in two different ways. $\endgroup$
    – sheppa28
    Nov 30, 2023 at 17:08
  • $\begingroup$ it's not hard to show that $\operatorname{E}(h_2^2) \geq {\left( \sigma_2 \right)}^2$ -- note that $\text{Var}(h_2) = E(h_2^{2})-E(h_2)^{2} \geq 0$, now sub in that $E(h_2)=\sigma_2$ and add $\sigma_2^2$ to both sides. Note that they're only equal when $\text{Var}(h_2)=0$ (consider: when does that happen?). Outside that, positively biased $\endgroup$
    – Glen_b
    Dec 1, 2023 at 1:41

1 Answer 1

1
$\begingroup$

For the normal distribution we should have $\theta_1$ as the MVUE because of https://en.wikipedia.org/wiki/Lehmann%E2%80%93Scheff%C3%A9_theorem

But note that $\theta_1 = $ is not in general an unbiased estimator of the fourth moment. It is for a normal distribution, but not in general. So that may be why it is not compared with $\theta_2$ when it is stated to be the minimum variance unbiased estimator.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.