# Need help finding UMVUE for a Poisson Distribution

My fellow classmates and I are stuck on a homework problem that is a three part problem to find the UMVUE of a Poisson distribution.

The problem goes like this:

Let X ~ Pois$$(\lambda$$), and we want to estimate $$\theta=e^{-\lambda}$$.

a) Find the Rao-Cramer lower bound for an unbiased estimator for $$\theta$$ (based on a sample of size n). Hint: The differentiation needs to be with respect to $$\theta$$, not $$\lambda$$.

So the majority of us used the Fisher Information Index to get:

$$I_1(\theta) = \Sigma ((\frac{d}{d\theta}ln(e^{-\lambda}\frac{\lambda^x}{x!}))^2*e^{-\lambda}\frac{\lambda^x}{x!})$$

We then substituted in our ln $$e^{-\lambda}$$ for $$\theta$$.

$$I_1(\theta) = \Sigma ((\frac{d}{d\theta}ln(\theta\frac{\lambda^x}{x!}))^2*e^{-\lambda}\frac{\lambda^x}{x!})$$

After that, we broke out the ln function:

$$I_1(\theta) = \Sigma ((\frac{d}{d\theta}ln(\theta) + ln(\lambda^x) - ln(x!))^2*e^{-\lambda}\frac{\lambda^x}{x!})$$

Then we took the derivative with respect to $$\theta$$.

$$I_1(\theta) = \Sigma ((\frac{1}{\theta})^2*e^{-\lambda}\frac{\lambda^x}{x!})$$

This is where we have a little difference of opinion because one of us came up with the final answer as being:

$$\frac{ne^{\lambda}}{\theta}$$

He took the final answer by cancelling one of the $$\theta$$s and then using the Taylor series to get $$e^{\lambda}$$

Where two of us got the answer as being:

$$\frac{n}{\theta^2}$$

We don't know which is right because either one could make sense.

If someone could please let us know on this, we would be grateful.

Part b we are also stuck on as well.

b) Show that $$T=\Sigma X_i$$ is a complete sufficient statistics for $$\theta=e^{-\lambda}$$

Now, from our notes, we have that a probability distribution belongs to an exponential family if it can be written in the form $$e^{A(x)B(\theta)+C(X)+D(\theta)}$$

So we start out and we get:

$$e^{-\lambda}\frac{\lambda^x}{x!}$$ $$=e^{-\lambda}e^{ln(\frac{\lambda^x}{x!})}$$ $$=e^{-\lambda}e^{xln(\lambda)-ln(x!)}$$

This is where we get stuck because we are not sure 1) where to put the T into this equation and 2) how are we supposed to relate this back to the theta.

So far, we have only come up with:

$$=e^{ln(\theta+\Sigma X_i ln(\lambda)-ln(\Sigma X_i!)}$$

We know we are probably completely and totally wrong on this.

When it comes to part c, of our equation, we are a bit stuck as to what to do.

Part c goes like:

If we use $$T^* = (\frac{n-1}{n})^{\Sigma X_i}$$ as an estimator for $$\theta=e^{-\lambda}$$, as the Rao-Blackwell Theorem indicated, combine your findings in Parts (a) and (b) to determine Var($$T^*$$).

We know that this is a UVMUE that can probably be solved by the Lehmann-Scheffe theorem.

The thing is that we just don't know how to really apply it or how we can use what we find from parts a and b to get Var($$T^*$$).

If anyone can help out with any parts of this problem, we will be very grateful because we are very, very, very stuck on this.

• There is a statistics website in the stackexchange system, your question might get a better reception there.
– Gerry Myerson
Apr 7, 2013 at 10:12
• Can I migrate this back to stackexange? I am getting 0 response here. Apr 10, 2013 at 19:48
• Forget about $\lambda$ and work with $\theta$, otherwise it is too messy and unclear. Dec 7, 2014 at 15:07
• stats.stackexchange.com/q/349942/119261 Nov 13, 2021 at 14:26

(a) As I mentioned in comment, you should focus on the parameter of interest $$\theta$$, it does not help of writing some expressions containing both $$\theta$$ and $$\lambda$$ at the same time. Following this, it is routine to get the log-likelihood (denote $$\sum X_i$$ by $$T$$ and omit terms which don't contain $$\theta$$) : $$\ell(\theta) = n\log\theta + T\log(-\log\theta)$$ Therefore, $$\ell'(\theta) = \frac{n}{\theta} + \frac{T}{\theta\log\theta}$$ $$\ell''(\theta) = -\frac{n}{\theta^2} - \frac{T(\log\theta + 1)}{(\theta\log\theta)^2}$$ As $$E_\theta(T) = -n\log\theta$$, it follows that the Fisher information is $$I(\theta) = -E_\theta(\ell''(\theta)) = -\frac{n}{\theta^2\log\theta}$$ Hence the C-R lower bound is given by $$1/I(\theta) = \boxed{-\theta^2\log(\theta)/n}$$. Since $$\theta = P_\theta(X_i = 0)$$, an unbiased estimate of $$\theta$$ could be the sample proportion which takes 0, namely, $$\hat{\theta} = \frac{\sum_{i = 1}^n \mathrm{I}\{X_i = 0\}}{n}.$$ Or even simpler, just take $$\hat{\theta} = \mathrm{I}\{X_1 = 0\}.$$

(b) This is routine application of Fisher's factorization theorem and the property of exponential family. Not hard if you write things clearly.

(c) You may verify that (recall $$T \sim \mathrm{Poisson}(n\lambda)$$, and here would be easier to go back to work with $$\lambda$$ but keep in mind $$\theta$$ and $$\lambda$$ is one-to-one correspondence): $$E_\theta(T^*) = \sum_{k = 0}^\infty \left(1 - \frac{1}{n}\right)^k e^{-n\lambda}\frac{(n\lambda)^k}{k!} = e^{-n\lambda}e^{n\lambda(1 - n^{-1})} = \theta.$$ Hence $$T^*$$ can be (also) used as an unbiased estimator of $$\theta$$ (in fact, it is the Rao-Blackwellization of $$\hat{\theta}$$, see the last paragraph of this answer).

In addition, \begin{align*} E_\theta((T^*)^2) = \sum_{k = 0}^\infty \left(1 - \frac{1}{n}\right)^{2k} e^{-n\lambda}\frac{(n\lambda)^k}{k!} = e^{-n\lambda}e^{n\lambda(1 - n^{-1})^2} = \theta^{2 - \frac{1}{n}}. \end{align*} It then follows that $$\mathrm{Var}_\theta(T^*) = \theta^{2 - \frac{1}{n}} - \theta^2.$$

One may verify that we indeed have the C-R lower bound is strictly less (as @81235 pointed out in the comment) than $$\mathrm{Var}_\theta(T^*)$$, as the consequence of the famous inequality \begin{align*} e^{\frac{\lambda}{n}} > 1 + \frac{\lambda}{n}. \end{align*}

On the other hand, $$\mathrm{Var}_\theta(T^*)$$ is indeed not bigger than $$\mathrm{Var}_\theta(\hat{\theta}) = P(X_1 = 0)(1 - P(X_1 = 0)) = e^{-\lambda}(1 - e^{-\lambda}) = \theta - \theta^2.$$ Together, we observe that

1. $$T^*$$ is the UMVUE for $$\theta$$ (as the Rao-Blackwell Theorem guarantees);
2. The UMVUE does not necessarily achieve the C-R lower bound.

Part (c) relates to part (a) and part (b) in "as Rao-Blackwell Theorem indicated", which is worthwhile to elaborate. Basically, we want to show that the Rao-Blackwellized estimate $$E(\hat{\theta} \mid T)$$ based on the the estimate $$\hat{\theta}$$ proposed in part (a) exactly yields $$T^*$$. Indeed, applying the classical distributional result \begin{align*} X_1 \mid X_1 + \cdots + X_n = t \sim \text{Binom}(t, n^{-1}), \end{align*} it follows that \begin{align*} E(\hat{\theta} \mid T) = P(X_1 = 0 \mid T) = (1 - n^{-1})^T = T^*, \end{align*} hence the form of $$T^*$$.

• About the variance, you might be wrong. Rao-Blackwell does not involve C-R lower bound. Sep 4, 2015 at 23:40
• Actually CR lower bound is strictly less than $Var(T^{*})$. Sep 4, 2015 at 23:56
• @81235 Thanks for pointing out my careless mistake. It has been a while. Now I corrected my answer for part (c). Jul 15, 2021 at 21:13

$g(X_1)=I_{(X_1=0)}=\begin{cases} 1, & \text{if$X_1=0$} \\ 0 & \text{otherwise} \\ \end{cases}$

• This is a bit short for an answer. Can you expand on this?
– Andy
Dec 7, 2014 at 13:30

$$x_1,...,x_n\sim Pois(\lambda)$$

(a)

We want to estimate $$\theta=e^{-\lambda}$$, which is exactly $$P(x_1=0)$$. We take $$T(x)=I\{x_1=0\}$$ as an estimator. It might look dumb, but it is an unbiased estimator: $$E[T(x)]=1\cdot P(x_1=0) + 0\cdot P(x_1 \neq 0)=P(x_1=0)=\frac{e^{-\lambda}\cdot\lambda^0}{0!}=e^{-\lambda}=\theta$$

Now, let's loot at the MSE: $$E[(T(x)-\lambda)^2]=E[(T(x)-\theta+\theta-\lambda)^2]=E[(T(x)-\theta)^2]-2E[(T(x)-\theta)(\theta-\lambda)]+E[(\theta-\lambda)^2]$$ $$T(x)$$ is an unbiased estimator of $$\theta$$ and thus $$2E[(T(x)-\theta)(\theta-\lambda)]$$ falls. Using the MSE decomposition we get $$b^2(T,\lambda)=(\theta-\lambda)^2$$, so the bias of $$T$$ w.r.t. $$\lambda$$ is simply $$\theta-\lambda$$.

Now, it is time for the mighty CRB: $$Var(T)=E[(T(x)-\theta)^2]-(\theta-\lambda)^2 \geq \frac{\left(1+\frac{d}{d\lambda}b(T,\lambda)\right)^2}{I(\lambda)}$$ a momentary pause: the Fisher information for the Poisson is $$I(\lambda)=\frac{1}{\lambda}$$, so: $$Var(T)\geq\lambda \left(1+\frac{d}{d\lambda}(\theta-\lambda)\right)^2$$ another pause: we were hinted to differentiate w.r.t. $$\theta$$, so we need do substitute $$\lambda=-log(\theta)$$ and substitute the derivation: $$\frac{db}{d\lambda}=\frac{db}{d\theta}\cdot\frac{d\theta}{d\lambda}=\frac{db}{d\theta}\cdot\frac{d}{d\lambda}(e^{-\lambda})=\frac{db}{d\theta}\cdot(-e^{-\lambda})=-\theta\cdot\frac{db}{d\theta}$$

So we substitute: $$Var(T)\geq -log(\theta) \left(1-\theta\frac{d}{d\theta}(\theta+log(\theta))\right)^2=-log(\theta) \left(1-\theta\left(1+\frac{1}{\theta}\right)\right)^2=-log(\theta) (1-\theta -1)^2$$ and finally get: $$Var(T)\geq-log(\theta)\theta^2$$

(b) $$L(x,\lambda)=\prod_{i=1}^{n}{\frac{e^{-\lambda}\lambda^{x_i}}{x_i!}}=\frac{1}{\prod{x_i!}}e^{-n\lambda}\lambda^{\sum{x_i}}=\frac{1}{\prod{x_i!}}exp(-n\lambda)exp(log(\lambda)\sum{x_i})$$ So by denoting $$\eta=log(\lambda), S(x)=\sum{x_i}, A(\eta)=n\lambda, h(x)=\frac{1}{\prod{x_i!}}$$ we get a representation of the likelihood function as an exponential family.

Reparametrizing using $$\theta$$ we get $$\eta=log(-log(\theta)), A(\eta)=-n log(\theta), S(x)=\sum{x_i}, h(x)=\frac{1}{\prod{x_i!}}$$, again en exponential family. This is important as we get that $$S(x)=\sum{x_i}$$ is a sufficient statistic due to properties of exponential families (Fisher-Neyman factorization theorem).

$$S(x)$$ is the sum of $$x_1,...,x_n\sim Pois(-log(\theta))$$, so $$S(x)$$ itself is a Poisson RV with parameter $$-nlog(\theta)$$. Now, let $$g(S(x))$$ be a function s.t $$E[g(S)]=0$$:

$$E[g(S)]=\theta^{-n}\sum_{s=0}^{\infty}{g(s)\cdot\frac{\left(-n log(\theta) \right)^s}{s!}}$$ The only term that can be zero is $$g(s)$$, so if $$E[g(S(x))]=0$$ we conclude that $$g(s)=0$$ for all values of $$s$$, hence $$S(x)=\sum{x_i}$$ is a complete statistic. QED!

(c)

This is almost cheating: The corollary from the Lehmann-Scheffé theorem is that if S is a complete sufficient statistic, then applying the R-B procedure would yield the UMVUE.

last momentary pause: we were told "combine your findings in Parts (a) and (b)", so let's do it

In part (a) we got $$Var(T)\geq-log(\theta)\theta^2$$, in part (b) we got that $$S(x)=\sum{x_i}$$ is complete sufficient statistic, R-B has yielded $$T^*$$, so $$Var(T^* )\geq-log(\theta)\theta^2$$. That's it.

PS It would be proper to thank Prof. Pavel Chigansky of HUJI, who has taught us this exact problem in his statistical inference course.

• @sirman canonical enough? Jul 14, 2021 at 13:19
• spelling of my name is wrong, anyway thanks! Jul 14, 2021 at 14:41