# MSE and MLE with Poisson distribution

Consider a situation which is modeled by the Poisson distribution, $P(X_i=k|\theta)=\displaystyle\frac{e^{-\theta}\theta^k}{k!}$. Find:

(a) The MLE of $\theta$.

(b) The MSE of the MLE.

(c) The approximate distribution of the MLE from the Central Limit Theorem.

Okay, so I think I solved (a):

$f(X_1,X_2,...,X_n|\theta)=\displaystyle\frac{e^{-n\theta}\theta^{\Sigma X_i}}{X_1!\cdot...\cdot X_n!}$

so, $\displaystyle\frac{d(lnf)}{\theta}=-n+\displaystyle\frac{\Sigma X_i}{\theta} = 0$

and therefore $\hat\mu=\displaystyle\frac{\Sigma_{i=1}^n X_i}{n}=\boxed{\bar X}$.

If this is correct, how should I go about solving part (b)? I mean, if the MLE that I calculated is unbiased, then isn't the MSE of it just its variance? So, would the answer just be $Var(\bar X)$? Thank you!

Given observed data $$\mathbf{x} = (x_1,...,x_n)$$ the log-likelihood is:

$$\ell_\mathbf{x}(\theta) = n ( \bar{x} \ln (\theta) - \theta ).$$

This has derivatives:

\begin{aligned} \frac{d \ell_\mathbf{x}}{d \theta}(\theta) &= n \bigg( \frac{\bar{x}}{\theta} - 1 \bigg), \\[6pt] \frac{d^2 \ell_\mathbf{x}}{d \theta^2}(\theta) &= - \frac{n \bar{x}}{\theta^2} <0. \\[6pt] \end{aligned}

This shows that the log-likelihood function is concave, so the MLE occurs at the unique critical point given by:

$$0 = \frac{d \ell_\mathbf{x}}{d \theta}(\hat{\theta}) \quad \quad \quad \implies \quad \quad \quad \hat{\theta} = \bar{x}.$$

Since $$\mathbb{E}(X) = \theta$$ and $$\mathbb{V}(X) = \theta^2$$, it follows from the central limit theorem that when $$n$$ is large, we have the approximate distribution $$\hat{\theta} \sim \text{N} ( \theta, \theta^2/n )$$. The estimator is unbiased, and so it has MSE given by:

\begin{aligned} \text{MSE}(\hat{\theta},\theta) = \mathbb{V} (\hat{\theta}) = \mathbb{V} (\bar{X} ) = \frac{\theta^2}{n}. \end{aligned}

• Thank you so much! So, for part (c), would my answer be $\displaystyle\frac{1}{\sigma_x\sqrt{2\pi}} e^{-\frac{x^2}{2Var(X)}}$? – Bing Feb 16 '17 at 5:34
As per central limit theorem $(\bar x- E(\bar x))/ \operatorname{Std}(\bar x)$ follows standard normal distribution with mean 0 and standard deviation of 1.
I.e. $(\bar x- E(\bar x))/ \operatorname{Std}(\bar x)$ follows $N(0,1)$ or equivalently $\bar x$ follows Normal with mean $E(\bar x)$ and variance $\operatorname{var}(\bar x)$.
Note that $E(\bar x) = \theta$ and as $\operatorname{Var}(x) = \theta$ for a Poisson distribution, $\operatorname{Var}(\bar x) = \theta/n$ which basically mean $\bar x$ follows normal distribution with mean $\theta$ and variance $\theta/n$.