2
$\begingroup$

How can I derive the distribution of $$\bar{X^2}\quad \text{when}\quad X\sim N \left( \theta, \sigma^2 \right) $$

The context of this question is an exercise requiring me to show that $\bar{X^2}- \frac{\sigma^2}{n} $ is an unbiased estimator of $\theta^2$ and afterwards find its efficiency. Thus I must find its variance as well.

Obviously $\bar{X} \sim N \left(\theta ,\frac{\sigma^2}{n} \right) $ but when it comes to its square I would not know.

I suppose that my only option is to derive the distribution of $\bar{X^2}$ using the CDF technique and repeatedly use change of variables. I tried that but it leads to a mess. I cannot discern the resulting distribution either

Is there perhaps a neater way to derive the distribution of $\bar{X^2}$?

$\endgroup$
7
  • $\begingroup$ Note that the right hand side of the distributional equality is a sequence of distributions, so you are actually asking for the distribution of $X_n^2$ rather than $X^2$. The unscaled $X_n$ is degenerate in the limit and collapses to a point, so in the limit, $X_n^2$ collapses to the square of that point. $\endgroup$ Commented Dec 27, 2013 at 13:23
  • $\begingroup$ $X$ is actually the mean of a sample having distribution $N(\theta, \sigma^2 )$. Let me fix that. $\endgroup$
    – JohnK
    Commented Dec 27, 2013 at 13:34
  • $\begingroup$ I don't think this is a standard distribution. See this. $\endgroup$ Commented Dec 27, 2013 at 13:38
  • $\begingroup$ @fgnu Okay, that was my initial idea to approach the exercise. All other suggestions are welcome of course. $\endgroup$
    – JohnK
    Commented Dec 27, 2013 at 13:40
  • $\begingroup$ @fgnu isn't the distribution just a (scaled) noncentral $\chi^2$? $\endgroup$
    – guy
    Commented Dec 27, 2013 at 19:52

1 Answer 1

6
$\begingroup$

Since we are looking at a sample mean we have that

$$\bar X_n \sim_{approx} N \left(\theta ,\frac{\sigma^2}{n} \right)$$ which holds for "large but finite $n$" -since if $n\rightarrow \infty$ the sample mean unscaled does not have a distribution, since it becomes a constant, as the comments noted. Then our conclusions will be approximate (and we keep $n$ fixed). We have

$$Z= \left[ \frac{\left(\bar X_n -\theta \right) \sqrt{n}} {\sigma} \right]^2 \sim_{approx} \chi^2 \left({1} \right) $$

Expand the functional form for $Z$:

$$Z= \frac n{\sigma^2} \Big (\bar X_n^2 -2\theta \bar X_n + \theta^2\Big) $$

$$\Rightarrow \bar X_n^2 =\frac {\sigma^2}{n}Z + 2\theta \bar X_n - \theta^2$$

We know the moments of $\bar X_n$, and we know the moments of $Z$, so we don't really need the distribution of $\bar X_n^2$. I guess the OP can take it up from here. One should not forget to consider what may be the covariance of $Z$ and $\bar X_n$.

ADDEDNDUM
(I am completing this answer after a I gave the OP time to work it through).

We want to consider the estimator for $\theta^2$, denote it $\hat h_n$, $$\hat h_n = \bar X_n^2- \frac{\sigma^2}{n}$$

UNBIASEDNESS

Its expected value is

$$ E(\hat h_n) = E\left(\bar X_n^2- \frac{\sigma^2}{n}\right) = E\left(\frac {\sigma^2}{n}Z + 2\theta \bar X_n - \theta^2- \frac{\sigma^2}{n}\right)$$

Since $E(Z) = 1$ and $E(\bar X) = \theta$ we obtain

$$E(\hat h_n) = \frac {\sigma^2}{n} + 2\theta^2 -\theta^2 - \frac {\sigma^2}{n} = \theta^2$$

So $\hat h_n$ is unbiased.

EFFICIENCY
The varinace of the estimator is $$\operatorname{Var}(\hat h_n) = \operatorname{Var}\left(\frac {\sigma^2}{n}Z + 2\theta \bar X_n - \theta^2- \frac{\sigma^2}{n}\right) $$

$$= \left(\frac {\sigma^2}{n}\right)^2\operatorname{Var}(Z) + \left(2\theta\right)^2\operatorname{Var}(\bar X_n) +2\frac {\sigma^2}{n}2\theta\operatorname{Cov}(Z,\bar X_n)$$

We have $\operatorname{Var}(Z) =2$ and $\operatorname{Var}(\bar X_n) = \frac {\sigma^2}{n}$, which gets us to

$$\operatorname{Var}(\hat h_n)=2\left(\frac {\sigma^2}{n}\right)^2 + 4\theta^2\frac {\sigma^2}{n}+ 4\theta\frac {\sigma^2}{n}\operatorname{Cov}(Z,\bar X_n)$$

For the Covariance we have

$$\operatorname{Cov}(Z,\bar X_n) = E(Z\bar X_n) - E(Z)E(\bar X_n) = E\left(\left[ \frac{\left(\bar X_n -\theta \right) \sqrt{n}} {\sigma} \right]^2\bar X_n\right) - 1\cdot \theta$$

$$= \frac {n}{\sigma^2}E\left(\bar X_n^3 -2\theta \bar X_n^2 + \theta^2\bar X_n\right) -\theta$$

We have $E(\bar X_n^2) = \frac {\sigma^2}{n} + \theta^2$, and $E(\bar X_n) = \theta$. Also, one can verify ($\bar X_n$ follows a normal distribution with non-zero mean) that $$E(\bar X_n^3) = \theta\left(\theta^2+3\frac {\sigma^2}{n}\right) = \theta^3+3\theta\frac {\sigma^2}{n}$$

So $$ \operatorname{Cov}(Z,\bar X_n) = \frac {n}{\sigma^2}\left(\theta^3+3\theta\frac {\sigma^2}{n} -2\theta \left(\frac {\sigma^2}{n} + \theta^2\right) + \theta^2\theta\right) -\theta =0$$

(what is the intuition behind this result that says that $Z$ and $\bar X_n$ are uncorrelated?)

So the the variance of the estimator (for "large" but finite $n$) is just

$$\operatorname{Var}(\hat h_n)=2\left(\frac {\sigma^2}{n}\right)^2 + 4\theta^2\frac {\sigma^2}{n}$$

Note: One could more quickly consult the tables for the raw moments of a normal distribution with non-zero mean, and derive the variance of the estimator directly from

$$\operatorname{Var}(\hat h_n) = E\left(\bar X_n^4\right) - \left(E(\bar X_n^2)\right)^2$$

...but one would then lose the conclusion that $Z$ and $\bar X_n$ are uncorrelated.

$\endgroup$
4
  • $\begingroup$ I have changed the notation a bit in my question. Plase see the edited one. $\endgroup$
    – JohnK
    Commented Dec 27, 2013 at 13:40
  • $\begingroup$ I doesn't make any difference except that you will have to accept the unavoidable approximation error that comes from the fact that the distribution of the sample mean is approximate and holds for "large samples" (since as the comments mentioned the sample mean collapses to a constant when the sample truly goes to infinity). So the relation with the chi-square distribution holds also approximately and then you can proceed with my suggestion. $\endgroup$ Commented Dec 27, 2013 at 13:45
  • $\begingroup$ Thank you very much. In order to find the efficiency of the estimator we additionally need the Cramer Rao bound that is most easily computed from the original distribution. Would you mind giving me your insight as to why those variables are uncorrelated? $\endgroup$
    – JohnK
    Commented Dec 28, 2013 at 10:05
  • 1
    $\begingroup$ The usual example of two "uncorrelated but dependent" rv's is to consider the square of a Uniform rv symmetric around zero.Here too, we have the square of a standard normal (a chi-square rv that is) that is also uncorrelated but dependent with its parent standard normal. This should permit us to understand a general rule (not unique of course) as to when we can have this phenomenon of purely non-linear dependence. $\endgroup$ Commented Dec 28, 2013 at 14:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.