Consider a situation which is modeled by the Poisson distribution, $P(X_i=k|\theta)=\displaystyle\frac{e^{-\theta}\theta^k}{k!}$. Find:

(a) The MLE of $\theta$.

(b) The MSE of the MLE.

(c) The approximate distribution of the MLE from the Central Limit Theorem.

Okay, so I think I solved (a):

$f(X_1,X_2,...,X_n|\theta)=\displaystyle\frac{e^{-n\theta}\theta^{\Sigma X_i}}{X_1!\cdot...\cdot X_n!}$

so, $\displaystyle\frac{d(lnf)}{\theta}=-n+\displaystyle\frac{\Sigma X_i}{\theta} = 0$

and therefore $\hat\mu=\displaystyle\frac{\Sigma_{i=1}^n X_i}{n}=\boxed{\bar X}$.

If this is correct, how should I go about solving part (b)? I mean, if the MLE that I calculated is unbiased, then isn't the MSE of it just its variance? So, would the answer just be $Var(\bar X)$? Thank you!


3 Answers 3


Given observed data $\mathbf{x} = (x_1,...,x_n)$ the log-likelihood is:

$$\ell_\mathbf{x}(\theta) = n ( \bar{x} \ln (\theta) - \theta ).$$

This has derivatives:

$$\begin{equation} \begin{aligned} \frac{d \ell_\mathbf{x}}{d \theta}(\theta) &= n \bigg( \frac{\bar{x}}{\theta} - 1 \bigg), \\[6pt] \frac{d^2 \ell_\mathbf{x}}{d \theta^2}(\theta) &= - \frac{n \bar{x}}{\theta^2} <0. \\[6pt] \end{aligned} \end{equation}$$

This shows that the log-likelihood function is concave, so the MLE occurs at the unique critical point given by:

$$0 = \frac{d \ell_\mathbf{x}}{d \theta}(\hat{\theta}) \quad \quad \quad \implies \quad \quad \quad \hat{\theta} = \bar{x}.$$

Since $\mathbb{E}(X) = \theta$ and $\mathbb{V}(X) = \theta$, it follows from the central limit theorem that when $n$ is large, we have the approximate distribution $\hat{\theta} \sim \text{N} ( \theta, \theta/n )$. The estimator is unbiased, and so it has MSE given by:

$$\begin{equation} \begin{aligned} \text{MSE}(\hat{\theta},\theta) = \mathbb{V} (\hat{\theta}) = \mathbb{V} (\bar{X} ) = \frac{\theta}{n}. \end{aligned} \end{equation}$$


Your answers and understanding is correct.

For the 3rd answer - the MLE would tend to follow a normal distribution as per as Central limit theorem.

  • $\begingroup$ Thank you so much! So, for part (c), would my answer be $\displaystyle\frac{1}{\sigma_x\sqrt{2\pi}} e^{-\frac{x^2}{2Var(X)}}$? $\endgroup$
    – Bing
    Feb 16, 2017 at 5:34

As per central limit theorem $(\bar x- E(\bar x))/ \operatorname{Std}(\bar x)$ follows standard normal distribution with mean 0 and standard deviation of 1.

I.e. $(\bar x- E(\bar x))/ \operatorname{Std}(\bar x)$ follows $N(0,1)$ or equivalently $\bar x$ follows Normal with mean $E(\bar x)$ and variance $\operatorname{var}(\bar x)$.

Note that $E(\bar x) = \theta$ and as $\operatorname{Var}(x) = \theta$ for a Poisson distribution, $\operatorname{Var}(\bar x) = \theta/n$ which basically mean $\bar x$ follows normal distribution with mean $\theta$ and variance $\theta/n$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.