# Normal distributed random sample: find the least variance from the set of all unbiased estimators of $\theta$

Let $$X_{1},X_{2},\ldots,X_{n}$$ be a random sample from $$X\sim\mathcal{N}(0,\sigma^{2})$$.

(a) Find the least variance from the set of all unbiased estimators of $$\sigma^{2}$$.

(b) Find a sufficient statistics of $$\sigma^{2}$$.

(c) Obtain from this statistics an unbiased estimator to $$\sigma^{2}$$.

(d) Verify if this estimator is efficient.

MY ATTEMPTS (EDITED)

(a) In the first place, let us determine the Fischer information of $$\sigma$$: \begin{align*} & f(x|\sigma) = \frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac{x^{2}}{2\sigma^{2}}\right) \Rightarrow \ln f(x|\sigma) = -\ln(\sigma) - \frac{\ln(2\pi)}{2} - \frac{x^{2}}{2\sigma^{2}} \Rightarrow\\\\ & \frac{\partial\ln f(x|\sigma)}{\partial\sigma} = -\frac{1}{\sigma} + \frac{x^{2}}{\sigma^{3}} \Rightarrow \frac{\partial^{2}\ln f(x|\sigma)}{\partial\sigma^{2}} = \frac{1}{\sigma^{2}} - \frac{3x^{2}}{\sigma^{4}} \Rightarrow\\\\ & I_{F}(\sigma) = \textbf{E}\left(-\frac{\partial^{2}\ln f(x|\sigma)}{\partial\sigma^{2}}\right) = -\frac{1}{\sigma^{2}} + \frac{3\textbf{E}(x^{2})}{\sigma^{4}} = -\frac{1}{\sigma^{2}} + \frac{3\sigma^{2}}{\sigma^{4}} = \frac{2}{\sigma^{2}} \end{align*}

because $$\textbf{E}(X) = 0$$. Thus $$\textbf{Var}(\hat{\sigma}) \geq \sigma^{2}/2n$$.

(b) Consider the likelihood function: \begin{align*} L(\textbf{x}|\sigma^{2}) = \prod_{k=1}^{n}f(x_{k}|\sigma^{2}) = \left(\frac{1}{\sigma\sqrt{2\pi}}\right)^{n}\exp\left(-\frac{1}{\sigma^{2}}\sum_{k=1}^{n}x^{2}_{k}\right) \end{align*}

According to the factorization criterion, we have the following sufficient statistcs:

\begin{align*} S(\textbf{x}) = \sum_{k=1}^{n}x^{2}_{k} \end{align*}

(c) I started with the following relation \begin{align*} \textbf{E}(S) = \textbf{E}\left(\sum_{k=1}^{n}X^{2}_{k}\right) = n\sigma^{2} \end{align*}

Therefore the statistics $$\hat{\sigma} = S/n$$ is unbiased.

(d) Observe that $$\textbf{Var}(X^{2}_{k}) = 2\sigma^{4}$$, which may be obtained from the moment generating function of $$X\sim\mathcal{N}(0,\sigma^{2})$$, that is given by \begin{align*} M_{X}(t) = \exp\left(\frac{t^{2}\sigma^{2}}{2}\right) \end{align*}

Consequently, the following relations hold \begin{align*} \textbf{Var}(\hat{\sigma}) = \textbf{Var}\left(\frac{1}{n}\sum_{k=1}^{n}X^{2}_{k}\right) = \frac{1}{n^{2}}\sum_{k=1}^{n}\textbf{Var}(X^{2}_{k}) = \frac{2\sigma^{4}}{n} \end{align*}

Finally, we have \begin{align*} e(\hat{\sigma}) = \frac{1}{\textbf{Var}(\hat{\sigma})nI_{F}(\sigma)} = \frac{1}{4\sigma^{2}} \end{align*}

• As for (c), you could have just found $E(S)=n\sigma^2$, giving the unbiased estimator $S/n$. – StubbornAtom Apr 11 '19 at 16:20
• Yes $\operatorname{Var}\left(\frac{1}{n}\sum X_i^2\right)=\frac{2\sigma^4}{n}$, but that should also be the answer to part (a) since it is also the CR lower bound. As a result, the answer to efficiency will also change. – StubbornAtom Apr 11 '19 at 20:27
• In particular, note the expression for the CR lower bound as you did not apply the formula correctly. You are considering unbiased estimator of $\sigma^2$, not $\sigma$. – StubbornAtom Apr 11 '19 at 20:36

Joint density of the sample $$\mathbf X=(X_1,X_2,\ldots,X_n)$$ is

$$f(\mathbf x\mid\sigma)=\frac{1}{(\sigma\sqrt{2\pi})^n}\exp\left[-\frac{1}{2\sigma^2}\sum_{i=1}^n x_i^2\right]\quad,\,\small \mathbf x\in\mathbb R^n\,,\,\sigma>0$$

By Factorisation theorem, a sufficient statistic for $$\sigma^2$$ is $$T(\mathbf X)=\sum_{i=1}^n X_i^2$$

Going back to the joint density, we have

\begin{align} \frac{\partial}{\partial\sigma}\ln f(\mathbf x\mid \sigma)&=\frac{-n}{\sigma}+\frac{1}{\sigma^3}\sum_{i=1}^n x_i^2 \\&=\frac{n}{\sigma^3}\left(\frac{1}{n}\sum_{i=1}^n x_i^2-\sigma^2\right) \end{align}

This is the condition of equality (see this related post) in the Cramér-Rao inequality, which directly shows that $$\frac{T}{n}$$ is the UMVUE of $$\sigma^2$$. This also suggests that it is the most efficient estimator of $$\sigma^2$$ within the unbiased class.

So the answer to part (a) is $$\operatorname{Var}\left(\frac{T}{n}\right)$$, which equals the Cramér-Rao bound given by

$$\operatorname{Var}\left(\frac{T}{n}\right)=\frac{\left[\frac{d}{d\sigma}(\sigma^2)\right]^2}{I(\sigma)}$$

, where $$I(\sigma)=-n\operatorname E\left[\frac{\partial^2}{\partial\sigma^2}\ln f (X_1\mid\sigma)\right]$$ is the Fisher information in the sample.

You should be able to finish calculating $$I(\sigma)$$ from the point you have stopped. And note that direct calculation of $$\operatorname{Var}\left(\frac{T}{n}\right)$$ is also possible.

• In the first place, thank you for the answer. After your considerations, I have edited my response. Can you check it out for me? Thanks! – user242554 Apr 11 '19 at 17:10