1
$\begingroup$

Let $X_{1},X_{2},\ldots,X_{n}$ be a random sample from $X\sim\mathcal{N}(0,\sigma^{2})$.

(a) Find the least variance from the set of all unbiased estimators of $\sigma^{2}$.

(b) Find a sufficient statistics of $\sigma^{2}$.

(c) Obtain from this statistics an unbiased estimator to $\sigma^{2}$.

(d) Verify if this estimator is efficient.

MY ATTEMPTS (EDITED)

(a) In the first place, let us determine the Fischer information of $\sigma$: \begin{align*} & f(x|\sigma) = \frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac{x^{2}}{2\sigma^{2}}\right) \Rightarrow \ln f(x|\sigma) = -\ln(\sigma) - \frac{\ln(2\pi)}{2} - \frac{x^{2}}{2\sigma^{2}} \Rightarrow\\\\ & \frac{\partial\ln f(x|\sigma)}{\partial\sigma} = -\frac{1}{\sigma} + \frac{x^{2}}{\sigma^{3}} \Rightarrow \frac{\partial^{2}\ln f(x|\sigma)}{\partial\sigma^{2}} = \frac{1}{\sigma^{2}} - \frac{3x^{2}}{\sigma^{4}} \Rightarrow\\\\ & I_{F}(\sigma) = \textbf{E}\left(-\frac{\partial^{2}\ln f(x|\sigma)}{\partial\sigma^{2}}\right) = -\frac{1}{\sigma^{2}} + \frac{3\textbf{E}(x^{2})}{\sigma^{4}} = -\frac{1}{\sigma^{2}} + \frac{3\sigma^{2}}{\sigma^{4}} = \frac{2}{\sigma^{2}} \end{align*}

because $\textbf{E}(X) = 0$. Thus $\textbf{Var}(\hat{\sigma}) \geq \sigma^{2}/2n$.

(b) Consider the likelihood function: \begin{align*} L(\textbf{x}|\sigma^{2}) = \prod_{k=1}^{n}f(x_{k}|\sigma^{2}) = \left(\frac{1}{\sigma\sqrt{2\pi}}\right)^{n}\exp\left(-\frac{1}{\sigma^{2}}\sum_{k=1}^{n}x^{2}_{k}\right) \end{align*}

According to the factorization criterion, we have the following sufficient statistcs:

\begin{align*} S(\textbf{x}) = \sum_{k=1}^{n}x^{2}_{k} \end{align*}

(c) I started with the following relation \begin{align*} \textbf{E}(S) = \textbf{E}\left(\sum_{k=1}^{n}X^{2}_{k}\right) = n\sigma^{2} \end{align*}

Therefore the statistics $\hat{\sigma} = S/n$ is unbiased.

(d) Observe that $\textbf{Var}(X^{2}_{k}) = 2\sigma^{4}$, which may be obtained from the moment generating function of $X\sim\mathcal{N}(0,\sigma^{2})$, that is given by \begin{align*} M_{X}(t) = \exp\left(\frac{t^{2}\sigma^{2}}{2}\right) \end{align*}

Consequently, the following relations hold \begin{align*} \textbf{Var}(\hat{\sigma}) = \textbf{Var}\left(\frac{1}{n}\sum_{k=1}^{n}X^{2}_{k}\right) = \frac{1}{n^{2}}\sum_{k=1}^{n}\textbf{Var}(X^{2}_{k}) = \frac{2\sigma^{4}}{n} \end{align*}

Finally, we have \begin{align*} e(\hat{\sigma}) = \frac{1}{\textbf{Var}(\hat{\sigma})nI_{F}(\sigma)} = \frac{1}{4\sigma^{2}} \end{align*}

$\endgroup$
  • $\begingroup$ As for (c), you could have just found $E(S)=n\sigma^2$, giving the unbiased estimator $S/n$. $\endgroup$ – StubbornAtom Apr 11 at 16:20
  • $\begingroup$ Yes $\operatorname{Var}\left(\frac{1}{n}\sum X_i^2\right)=\frac{2\sigma^4}{n}$, but that should also be the answer to part (a) since it is also the CR lower bound. As a result, the answer to efficiency will also change. $\endgroup$ – StubbornAtom Apr 11 at 20:27
  • $\begingroup$ In particular, note the expression for the CR lower bound as you did not apply the formula correctly. You are considering unbiased estimator of $\sigma^2$, not $\sigma$. $\endgroup$ – StubbornAtom Apr 11 at 20:36
1
$\begingroup$

Joint density of the sample $\mathbf X=(X_1,X_2,\ldots,X_n)$ is

$$f(\mathbf x\mid\sigma)=\frac{1}{(\sigma\sqrt{2\pi})^n}\exp\left[-\frac{1}{2\sigma^2}\sum_{i=1}^n x_i^2\right]\quad,\,\small \mathbf x\in\mathbb R^n\,,\,\sigma>0$$

By Factorisation theorem, a sufficient statistic for $\sigma^2$ is $$ T(\mathbf X)=\sum_{i=1}^n X_i^2$$

Going back to the joint density, we have

\begin{align} \frac{\partial}{\partial\sigma}\ln f(\mathbf x\mid \sigma)&=\frac{-n}{\sigma}+\frac{1}{\sigma^3}\sum_{i=1}^n x_i^2 \\&=\frac{n}{\sigma^3}\left(\frac{1}{n}\sum_{i=1}^n x_i^2-\sigma^2\right) \end{align}

This is the condition of equality (see this related post) in the Cramér-Rao inequality, which directly shows that $\frac{T}{n}$ is the UMVUE of $\sigma^2$. This also suggests that it is the most efficient estimator of $\sigma^2$ within the unbiased class.

So the answer to part (a) is $\operatorname{Var}\left(\frac{T}{n}\right)$, which equals the Cramér-Rao bound given by

$$\operatorname{Var}\left(\frac{T}{n}\right)=\frac{\left[\frac{d}{d\sigma}(\sigma^2)\right]^2}{I(\sigma)}$$

, where $I(\sigma)=-n\operatorname E\left[\frac{\partial^2}{\partial\sigma^2}\ln f (X_1\mid\sigma)\right]$ is the Fisher information in the sample.

You should be able to finish calculating $I(\sigma)$ from the point you have stopped. And note that direct calculation of $\operatorname{Var}\left(\frac{T}{n}\right)$ is also possible.

$\endgroup$
  • $\begingroup$ In the first place, thank you for the answer. After your considerations, I have edited my response. Can you check it out for me? Thanks! $\endgroup$ – user1337 Apr 11 at 17:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.