Let $X_{1},X_{2},\ldots,X_{n}$ represent a random sample whose distribution is given by $X\sim\mathcal{N}(0,\theta)$. Find the MLE of $\theta$. Then obtain the score function, find the Fischer information and prove the MLE is an efficient estimator for $\theta$.


Firstly, let us determine the MLE estimator \begin{align*} & p(x|\theta) = \frac{1}{\sqrt{2\pi\theta}}\exp\left\{-\frac{x^{2}}{2\theta}\right\} \Rightarrow\\ & p(\textbf{x}|\theta) = \left(2\pi\theta\right)^{-n/2}\exp\left\{-\frac{1}{2\theta}\sum_{i=1}^{n}x^{2}_{i}\right\}\Rightarrow\\\\ & \ln p(\textbf{x}|\theta) = -\frac{n}{2}\ln(2\pi) - \frac{n}{2}\ln(\theta) - \frac{1}{2\theta}\sum_{i=1}^{n}x^{2}_{i} \Rightarrow\\ &\frac{\partial\ln p(\textbf{x}|\theta)}{\partial\theta} = -\frac{n}{2\theta} + \frac{1}{2\theta^{2}}\sum_{i=1}^{n}x^{2}_{i} = 0 \Rightarrow \hat{\theta} = \frac{1}{n}\sum_{i=1}^{n}x^{2}_{i} \end{align*}

From then on I get stuck. This is what I have tried: \begin{align*} \frac{\partial^{2}\ln p(\textbf{x}|\theta)}{\partial\theta^{2}} = \frac{n}{2\theta^{2}} - \frac{1}{\theta^{3}}\sum_{i=1}^{n}x^{2}_{i} \Rightarrow I(\theta) = -\frac{n}{2\theta^{2}} + \frac{1}{\theta^{3}}\sum_{i=1}^{n}\textbf{E}(X^{2}_{i}) \end{align*}

Unfortunately, this leads to nowhere, even though I know that $\textbf{E}(X^{2}_{i}) = \theta$. Can someone help me out with the Fischer Information and $\textbf{Var}(X^{2}_{i})$?

  • $\begingroup$ For fisher information $\sum_{i=1}^{n}\textbf{E}(X^{2}_{i}) = n\theta$. For $ \textbf{Var}(X^{2}_{i})$, find $\textbf{Var}(X^{4}_{i})$ by searching normal distribution. $\endgroup$ – user158565 May 30 at 4:26
  • $\begingroup$ $ Var(X_i^4)$ should be $E(X_i^4)$. $\endgroup$ – user158565 May 30 at 4:42

To establish efficiency, you need to compare the variance of your estimator with the Cramér-Rao bound. You have already derived an expression for the Fisher information, so now you just need to simplify it. Continuing what you have already found, you have $\mathbb{E}(X_i^2) = \theta$ which implies that the estimator is unbiased. Thus, using your existing expression, the Fisher information can be written as:

$$\begin{equation} \begin{aligned} \mathcal{I}(\theta) &= \mathbb{E} \bigg( \frac{n}{\theta^3} \Bigg[ \frac{1}{n} \sum_{i=1}^n X_i^2 - \frac{\theta}{2} \Bigg] \bigg) \\[6pt] &= \mathbb{E} \bigg( \frac{n}{\theta^3} \Bigg[ \hat{\theta} - \frac{\theta}{2} \Bigg] \bigg) \\[6pt] &= \frac{n}{\theta^3} \Bigg[ \mathbb{E}(\hat{\theta}) - \frac{\theta}{2} \Bigg] \\[6pt] &= \frac{n}{\theta^3} \Bigg[ \theta - \frac{\theta}{2} \Bigg] \\[6pt] &= \frac{n}{2 \theta^2}. \\[6pt] \end{aligned} \end{equation}$$

Since $\hat{\theta}$ is an unbiased estimator of $\theta$, the Cramér-Rao bound is:

$$\mathbb{V}(\hat{\theta}) \geqslant \mathcal{I}(\theta)^{-1} = \frac{2}{n} \cdot \theta^2.$$

To determine whether you have an efficient estimator, you need to establish whether or not the variance of the estimator achieves this lower bound. To do this, you will have to write out the variance of your estimator, and simplify this variance expression. You will find that this expression depends on the fourth moment of the normal distribution, and since the normal distribution is mesokurtic, this expression simplifies easily.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.