Estimator for $E[X]^2$

Question

I'm trying to understand the theory of estimators. As I understand it now, if you have an r.v. $X$ and take $n$ i.i.d. samples then an estimator for $E[X^{2}]$ would be $\overline{X^{2}}$ since $E[\overline{X^{2}}] = E[X^{2}]$ (probably only true for some kind of "nice" r.v.).

However, the same kind of nice result doesn't occur when trying to estimate $E[X]^{2}$. That is to say, the function $\overline{X}^{2}$ does not estimate this. But I'm not sure I understand which function does estimate it. I have the equation $V[\overline{X}] = E[\overline{X}^{2}]-E[\overline{X}]^{2}$ and so $E[X]^{2} = E[\overline{X}^{2}]-V[\overline{X}]$. This seems relevant but I'm not sure what to conclude from this.

Answer 1

By continuous mapping theorem $\bar{X}^2 \to \text{E}[X]^2$ in probability, so I would say it is a good estimate.

Depending on the distribution of $X$, if $\bar{X}$ is the MLE of $\text{E}[X]$, then $\bar{X}^2$ will be the MLE of $\text{E}[X]^2$ (MLE is invariant to transformation).

If $X_i$ are iid and $\text{Var}[X] = \sigma^2$, then $\text{Var}[\bar{X}] = \text{Var}\left[ \dfrac{1}{n} \sum_{i=1}^n X_i\right] = \dfrac{1}{n^2}\sum_{i=1}^n \text{Var}[X_i] = \dfrac{\sigma^2}{n}$

Answer 2

Background: unbiased estimators of products of population moments

If you desire an UNBIASED estimator of a (product of moments), there are 3 varieties:

1. Polykays (a generalisation of k-statistics): these are unbiased estimators of products of population cumulants. The term polykay was coined by Tukey, but the concept goes back to Dressel (1940).

2. Polyaches (a generalisation of h-statistics): these are unbiased estimators of products of population central moments. i.e.

$$E\left[\text{h}_{\{r,t,\ldots ,v\}}\right] = {\mu }_r {\mu }_t \cdots {\mu }_v\text{$\, $}$$ ...... where ${\mu }_r$ denotes the $r^{th}$ central moment of the population.

3. Polyraws: these are unbiased estimators of products of population raw moments. That is, you wish to find the $polyraw_{r, t, ...v}$ such that:

$$E\left[\text{polyraw}_{\{r,t,\ldots ,v\}}\right] = \acute{\mu }_r \acute{\mu }_t \cdots \acute{\mu }_v\text{$\, $}$$

...... where $\acute{\mu }_r$ denotes the $r^{th}$ raw moment of the population.

---

The Problem

We are given a random sample $(X_1, X_2, \dots, X_n)$ drawn on parent random variable $X$.

If we desire an unbiased estimator of: $(E[X])^2 = \acute{\mu }_1 \acute{\mu }_1$, then an unbiased estimator is the {1,1} polyraw:

where $s_r = \sum_{i=1}^n X_i^r$ denotes the $r^{th}$ power sum.

---

Comparison

Benjamin proposed the estimator: $\bar{X}^2 = (\frac{s_1}{n})^2$. This is not an unbiased estimator, since $E[(\frac{s_1}{n})^2]$ is just the $1^{st}$ RawMoment of $(\frac{s_1}{n})^2$:

which is not equal to $\acute{\mu }_1^2$.

Let us check the polyraw solution:

... which is an unbiased estimator.

Plainly, unbiasedness is not everything, and we could equally calculate, for example, the MSE (mean-squared error) of each estimator using exactly the same tools.

[Update: Just had a quick play with this: in a simple test case of $X \sim N(0,\sigma^2)$, the polyraw unbiased estimator has smaller MSE than Ben's ML estimator, for all sample sizes $n$. That is, at least for the test case of Normality, the polyraw unbiased estimator dominates the maximum likelihood estimator, at all sample sizes. ]

Notes

• PolyRaw, RawMomentToRaw etc are functions in the mathStatica package for Mathematica

• I confess to the neologism polyache in our Springer book (2002) (and more recently, to polyraw in the latest edition).