MSE and MLE with Poisson distribution

Question

Consider a situation which is modeled by the Poisson distribution, $P(X_i=k|\theta)=\displaystyle\frac{e^{-\theta}\theta^k}{k!}$. Find:

(a) The MLE of $\theta$.

(b) The MSE of the MLE.

(c) The approximate distribution of the MLE from the Central Limit Theorem.

---

Okay, so I think I solved (a):

$f(X_1,X_2,...,X_n|\theta)=\displaystyle\frac{e^{-n\theta}\theta^{\Sigma X_i}}{X_1!\cdot...\cdot X_n!}$

so, $\displaystyle\frac{d(lnf)}{\theta}=-n+\displaystyle\frac{\Sigma X_i}{\theta} = 0$

and therefore $\hat\mu=\displaystyle\frac{\Sigma_{i=1}^n X_i}{n}=\boxed{\bar X}$.

If this is correct, how should I go about solving part (b)? I mean, if the MLE that I calculated is unbiased, then isn't the MSE of it just its variance? So, would the answer just be $Var(\bar X)$? Thank you!

Answer 1

Given observed data $\mathbf{x} = (x_1,...,x_n)$ the log-likelihood is:

$$\ell_\mathbf{x}(\theta) = n ( \bar{x} \ln (\theta) - \theta ).$$

This has derivatives:

$$\begin{equation} \begin{aligned} \frac{d \ell_\mathbf{x}}{d \theta}(\theta) &= n \bigg( \frac{\bar{x}}{\theta} - 1 \bigg), \\[6pt] \frac{d^2 \ell_\mathbf{x}}{d \theta^2}(\theta) &= - \frac{n \bar{x}}{\theta^2} <0. \\[6pt] \end{aligned} \end{equation}$$

This shows that the log-likelihood function is concave, so the MLE occurs at the unique critical point given by:

$$0 = \frac{d \ell_\mathbf{x}}{d \theta}(\hat{\theta}) \quad \quad \quad \implies \quad \quad \quad \hat{\theta} = \bar{x}.$$

Since $\mathbb{E}(X) = \theta$ and $\mathbb{V}(X) = \theta$, it follows from the central limit theorem that when $n$ is large, we have the approximate distribution $\hat{\theta} \sim \text{N} ( \theta, \theta/n )$. The estimator is unbiased, and so it has MSE given by:

$$\begin{equation} \begin{aligned} \text{MSE}(\hat{\theta},\theta) = \mathbb{V} (\hat{\theta}) = \mathbb{V} (\bar{X} ) = \frac{\theta}{n}. \end{aligned} \end{equation}$$

Answer 2

Your answers and understanding is correct.

For the 3rd answer - the MLE would tend to follow a normal distribution as per as Central limit theorem.

Answer 3

As per central limit theorem $(\bar x- E(\bar x))/ \operatorname{Std}(\bar x)$ follows standard normal distribution with mean 0 and standard deviation of 1.

I.e. $(\bar x- E(\bar x))/ \operatorname{Std}(\bar x)$ follows $N(0,1)$ or equivalently $\bar x$ follows Normal with mean $E(\bar x)$ and variance $\operatorname{var}(\bar x)$.

Note that $E(\bar x) = \theta$ and as $\operatorname{Var}(x) = \theta$ for a Poisson distribution, $\operatorname{Var}(\bar x) = \theta/n$ which basically mean $\bar x$ follows normal distribution with mean $\theta$ and variance $\theta/n$.