Let $X_1,X_2,....,X_n$ be a random sample of size $n$ from a population with cdf $F()$. Let $E(X)=\mu$ exist. Then estimate $\mu^2$ unbiasedly for the following three cases:-

(i) $Var(X)=\sigma^2$ exists and is known.

(ii) $Var(X)=\sigma^2$ exists and is unknown.

(iii) $Var(X)$ does not exist.

Now for (i) & (ii) I have obtained the following solutions:- (i) We define $$\bar X=\frac1n\sum_{i=1}^n X_i$$ The UE of $\mu^2$ is given as follows $$\bar X^2-\frac{\sigma^2}{n}$$ (ii) We define $$S^2=\frac{1}{n-1}\sum_{i=1}^n (X_i-\bar X)^2$$ The UE of $\mu^2$ is given as follows $$\bar X^2-\frac{S^2}{n}$$

For (iii) I need help. And please let me know if my answers thus far are correct. Thanks.

  • 3
    $\begingroup$ Nice question (+1). When $n\ge 2,$ I wonder what $E[X_1X_2]$ might be? (See stat.wisc.edu/~doksum/STAT709/n709-33.pdf formula (2) page 1 and Example 3.11 page 3 for instance.) $\endgroup$ – whuber Jun 14 '18 at 18:39
  • $\begingroup$ I have a question: If the variance does not exist but the mean does exist, this implies that the mean is infinite, doesn`t it? As $X^2>0$ we know that $E[X^2]$ exists (but is possibly infinite). However $Var(X)= E[X^2]-E[X]^2$. So it is impossible for the mean to be finite, as then the variance would exist. Is this argument wrong somewhere? $\endgroup$ – Sebastian Oct 11 '18 at 19:24
  • $\begingroup$ @Sebastian When we say the variance does not exist, we mean it is infinite. It is possible to have infinite variance but finite mean. $\endgroup$ – Robin Ryder Oct 11 '18 at 19:30
  • 1
    $\begingroup$ @Sebastian The classic example of having a finite mean but infinite variance is the $t$ distribution with $1 < \nu \le 2$ degrees of freedom. $\endgroup$ – Sycorax Oct 11 '18 at 19:31
  • $\begingroup$ Thank you that clarifies matters. I am aware that distributions can have finite mean but infinite variance but my professor used to keep the terms "no variance" and "infinite variance" seperate. $\endgroup$ – Sebastian Oct 11 '18 at 19:33

Only the third question remains to be answered, the case where $X$ has infinite variance.

When $n \gt 1,$ you can split the data into two smaller nonoverlapping (and therefore independent) samples, estimate $\mu$ separately in each subsample, and multiply the estimates. The independence assures the expectation of that product is the product of the expectations, so if each one of the estimates of $\mu$ is unbiased, so is your product estimate.

The simplest form of this idea is to let $X_i$ be one subsample of size $1$ and $X_j$ (for $j\ne i$) a different subsample of size $1.$ Using the estimator of $\mu$ in the question (for the case $n=1$) gives the estimator

$$t_{ij}(\mathbf{X}) = \left(\frac{1}{1} X_i\right) \left(\frac{1}{1} X_j\right) = X_iX_j.$$

Clearly $t_{ij}$ is unbiased because

$$\mathbb{E}(t_{ij}(\mathbf{X})) = \mathbb{E}(X_iX_j) = \mathbb{E}(X_i)\mathbb{E}(X_j) = \mu^2.$$

We can go further. Intuitively, this approach ignores a lot of information available in the sample. The theory of U statistics is based on generating all possible estimates $t_{ij}, 1\le i \lt j \le n,$ and averaging them:

$$U(\mathbf{X}) = \frac{1}{\binom{n}{2}}\sum_{1 \le i \lt j \le n} t_{ij}(\mathbf X) = \frac{1}{\binom{n}{2}}\sum_{1 \le i \lt j \le n} X_iX_j.$$

(The linearity of expectation shows this average remains unbiased.) Computations of variances show that when the underlying variance is finite, the "U statistic" has smaller variance than any individual $t_{ij}.$ (You might enjoy carrying out this approach for parts (1) and (2) of the question, because it leads directly and easily to the solutions given.) It would seem, however, that all bets are off when the underlying variance is infinite. Indeed, $t_{ij}$ may tend to vary less than the U statistic.

Simulations with power-law tails suggest the U-statistic approach still has merit. The estimates produced by any individual $t_{ij}$ tend to be less extreme than the U statistic, because they have less of a chance of sampling the occasional whopping big outlier that such distributions produce. Consequently, there's potentially a high risk in any given application that a $t_{ij}$ will grossly underestimate $\mu^2.$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.