6
$\begingroup$

My fellow classmates and I are stuck on a homework problem that is a three part problem to find the UMVUE of a Poisson distribution.

The problem goes like this:

Let X ~ Pois$(\lambda$), and we want to estimate $\theta=e^{-\lambda}$.

a) Find the Rao-Cramer lower bound for an unbiased estimator for $\theta$ (based on a sample of size n). Hint: The differentiation needs to be with respect to $\theta$, not $\lambda$.

So the majority of us used the Fisher Information Index to get:

$I_1(\theta) = \Sigma ((\frac{d}{d\theta}ln(e^{-\lambda}\frac{\lambda^x}{x!}))^2*e^{-\lambda}\frac{\lambda^x}{x!})$

We then substituted in our ln $e^{-\lambda}$ for $\theta$.

$I_1(\theta) = \Sigma ((\frac{d}{d\theta}ln(\theta\frac{\lambda^x}{x!}))^2*e^{-\lambda}\frac{\lambda^x}{x!})$

After that, we broke out the ln function:

$I_1(\theta) = \Sigma ((\frac{d}{d\theta}ln(\theta) + ln(\lambda^x) - ln(x!))^2*e^{-\lambda}\frac{\lambda^x}{x!})$

Then we took the derivative with respect to $\theta$.

$I_1(\theta) = \Sigma ((\frac{1}{\theta})^2*e^{-\lambda}\frac{\lambda^x}{x!})$

This is where we have a little difference of opinion because one of us came up with the final answer as being:

$\frac{ne^{\lambda}}{\theta}$

He took the final answer by cancelling one of the $\theta$s and then using the Taylor series to get $e^{\lambda}$

Where two of us got the answer as being:

$\frac{n}{\theta^2}$

We don't know which is right because either one could make sense.

If someone could please let us know on this, we would be grateful.

Part b we are also stuck on as well.

b) Show that $T=\Sigma X_i$ is a complete sufficient statistics for $\theta=e^{-\lambda}$

Now, from our notes, we have that a probability distribution belongs to an exponential family if it can be written in the form $e^{A(x)B(\theta)+C(X)+D(\theta)}$

So we start out and we get:

$e^{-\lambda}\frac{\lambda^x}{x!}$ $=e^{-\lambda}e^{ln(\frac{\lambda^x}{x!})}$ $=e^{-\lambda}e^{xln(\lambda)-ln(x!)}$

This is where we get stuck because we are not sure 1) where to put the T into this equation and 2) how are we supposed to relate this back to the theta.

So far, we have only come up with:

$=e^{ln(\theta+\Sigma X_i ln(\lambda)-ln(\Sigma X_i!)}$

We know we are probably completely and totally wrong on this.

When it comes to part c, of our equation, we are a bit stuck as to what to do.

Part c goes like:

If we use $T^* = (\frac{n-1}{n})^{\Sigma X_i}$ as an estimator for $\theta=e^{-\lambda}$, as the Rao-Blackwell Theorem indicated, combine your findings in Parts (a) and (b) to determine Var($T^*$).

We know that this is a UVMUE that can probably be solved by the Lehmann-Scheffe theorem.

The thing is that we just don't know how to really apply it or how we can use what we find from parts a and b to get Var($T^*$).

If anyone can help out with any parts of this problem, we will be very grateful because we are very, very, very stuck on this.

$\endgroup$
4
  • 1
    $\begingroup$ There is a statistics website in the stackexchange system, your question might get a better reception there. $\endgroup$
    – Gerry Myerson
    Apr 7, 2013 at 10:12
  • $\begingroup$ Can I migrate this back to stackexange? I am getting 0 response here. $\endgroup$
    – Perdue
    Apr 10, 2013 at 19:48
  • $\begingroup$ Forget about $\lambda$ and work with $\theta$, otherwise it is too messy and unclear. $\endgroup$
    – Zhanxiong
    Dec 7, 2014 at 15:07
  • $\begingroup$ stats.stackexchange.com/q/349942/119261 $\endgroup$ Nov 13, 2021 at 14:26

3 Answers 3

8
$\begingroup$

(a) As I mentioned in comment, you should focus on the parameter of interest $\theta$, it does not help of writing some expressions containing both $\theta$ and $\lambda$ at the same time. Following this, it is routine to get the log-likelihood (denote $\sum X_i$ by $T$ and omit terms which don't contain $\theta$) : $$\ell(\theta) = n\log\theta + T\log(-\log\theta)$$ Therefore, $$\ell'(\theta) = \frac{n}{\theta} + \frac{T}{\theta\log\theta}$$ $$\ell''(\theta) = -\frac{n}{\theta^2} - \frac{T(\log\theta + 1)}{(\theta\log\theta)^2}$$ As $E_\theta(T) = -n\log\theta$, it follows that the Fisher information is $$I(\theta) = -E_\theta(\ell''(\theta)) = -\frac{n}{\theta^2\log\theta}$$ Hence the C-R lower bound is given by $1/I(\theta) = \boxed{-\theta^2\log(\theta)/n}$. Since $\theta = P_\theta(X_i = 0)$, an unbiased estimate of $\theta$ could be the sample proportion which takes 0, namely, $$\hat{\theta} = \frac{\sum_{i = 1}^n \mathrm{I}\{X_i = 0\}}{n}.$$ Or even simpler, just take $$\hat{\theta} = \mathrm{I}\{X_1 = 0\}.$$

(b) This is routine application of Fisher's factorization theorem and the property of exponential family. Not hard if you write things clearly.

(c) You may verify that (recall $T \sim \mathrm{Poisson}(n\lambda)$, and here would be easier to go back to work with $\lambda$ but keep in mind $\theta$ and $\lambda$ is one-to-one correspondence): $$E_\theta(T^*) = \sum_{k = 0}^\infty \left(1 - \frac{1}{n}\right)^k e^{-n\lambda}\frac{(n\lambda)^k}{k!} = e^{-n\lambda}e^{n\lambda(1 - n^{-1})} = \theta.$$ Hence $T^*$ can be (also) used as an unbiased estimator of $\theta$ (in fact, it is the Rao-Blackwellization of $\hat{\theta}$, see the last paragraph of this answer).

In addition, \begin{align*} E_\theta((T^*)^2) = \sum_{k = 0}^\infty \left(1 - \frac{1}{n}\right)^{2k} e^{-n\lambda}\frac{(n\lambda)^k}{k!} = e^{-n\lambda}e^{n\lambda(1 - n^{-1})^2} = \theta^{2 - \frac{1}{n}}. \end{align*} It then follows that $$\mathrm{Var}_\theta(T^*) = \theta^{2 - \frac{1}{n}} - \theta^2.$$

One may verify that we indeed have the C-R lower bound is strictly less (as @81235 pointed out in the comment) than $\mathrm{Var}_\theta(T^*)$, as the consequence of the famous inequality \begin{align*} e^{\frac{\lambda}{n}} > 1 + \frac{\lambda}{n}. \end{align*}

On the other hand, $\mathrm{Var}_\theta(T^*)$ is indeed not bigger than $$\mathrm{Var}_\theta(\hat{\theta}) = P(X_1 = 0)(1 - P(X_1 = 0)) = e^{-\lambda}(1 - e^{-\lambda}) = \theta - \theta^2.$$ Together, we observe that

  1. $T^*$ is the UMVUE for $\theta$ (as the Rao-Blackwell Theorem guarantees);
  2. The UMVUE does not necessarily achieve the C-R lower bound.

Part (c) relates to part (a) and part (b) in "as Rao-Blackwell Theorem indicated", which is worthwhile to elaborate. Basically, we want to show that the Rao-Blackwellized estimate $E(\hat{\theta} \mid T)$ based on the the estimate $\hat{\theta}$ proposed in part (a) exactly yields $T^*$. Indeed, applying the classical distributional result \begin{align*} X_1 \mid X_1 + \cdots + X_n = t \sim \text{Binom}(t, n^{-1}), \end{align*} it follows that \begin{align*} E(\hat{\theta} \mid T) = P(X_1 = 0 \mid T) = (1 - n^{-1})^T = T^*, \end{align*} hence the form of $T^*$.

$\endgroup$
3
  • 2
    $\begingroup$ About the variance, you might be wrong. Rao-Blackwell does not involve C-R lower bound. $\endgroup$
    – 81235
    Sep 4, 2015 at 23:40
  • 1
    $\begingroup$ Actually CR lower bound is strictly less than $Var(T^{*})$. $\endgroup$
    – 81235
    Sep 4, 2015 at 23:56
  • $\begingroup$ @81235 Thanks for pointing out my careless mistake. It has been a while. Now I corrected my answer for part (c). $\endgroup$
    – Zhanxiong
    Jul 15, 2021 at 21:13
0
$\begingroup$

How about the indicator function:

$g(X_1)=I_{(X_1=0)}=\begin{cases} 1, & \text{if $X_1=0$ } \\ 0 & \text{otherwise} \\ \end{cases}$

$\endgroup$
1
  • 3
    $\begingroup$ This is a bit short for an answer. Can you expand on this? $\endgroup$
    – Andy
    Dec 7, 2014 at 13:30
0
$\begingroup$

$x_1,...,x_n\sim Pois(\lambda)$

(a)

We want to estimate $\theta=e^{-\lambda}$, which is exactly $P(x_1=0)$. We take $T(x)=I\{x_1=0\}$ as an estimator. It might look dumb, but it is an unbiased estimator: $$E[T(x)]=1\cdot P(x_1=0) + 0\cdot P(x_1 \neq 0)=P(x_1=0)=\frac{e^{-\lambda}\cdot\lambda^0}{0!}=e^{-\lambda}=\theta$$

Now, let's loot at the MSE: $$E[(T(x)-\lambda)^2]=E[(T(x)-\theta+\theta-\lambda)^2]=E[(T(x)-\theta)^2]-2E[(T(x)-\theta)(\theta-\lambda)]+E[(\theta-\lambda)^2]$$ $T(x)$ is an unbiased estimator of $\theta$ and thus $2E[(T(x)-\theta)(\theta-\lambda)]$ falls. Using the MSE decomposition we get $b^2(T,\lambda)=(\theta-\lambda)^2$, so the bias of $T$ w.r.t. $\lambda$ is simply $\theta-\lambda$.

Now, it is time for the mighty CRB: $$Var(T)=E[(T(x)-\theta)^2]-(\theta-\lambda)^2 \geq \frac{\left(1+\frac{d}{d\lambda}b(T,\lambda)\right)^2}{I(\lambda)}$$ a momentary pause: the Fisher information for the Poisson is $I(\lambda)=\frac{1}{\lambda}$, so: $$Var(T)\geq\lambda \left(1+\frac{d}{d\lambda}(\theta-\lambda)\right)^2$$ another pause: we were hinted to differentiate w.r.t. $\theta$, so we need do substitute $\lambda=-log(\theta)$ and substitute the derivation: $$\frac{db}{d\lambda}=\frac{db}{d\theta}\cdot\frac{d\theta}{d\lambda}=\frac{db}{d\theta}\cdot\frac{d}{d\lambda}(e^{-\lambda})=\frac{db}{d\theta}\cdot(-e^{-\lambda})=-\theta\cdot\frac{db}{d\theta}$$

So we substitute: $$Var(T)\geq -log(\theta) \left(1-\theta\frac{d}{d\theta}(\theta+log(\theta))\right)^2=-log(\theta) \left(1-\theta\left(1+\frac{1}{\theta}\right)\right)^2=-log(\theta) (1-\theta -1)^2$$ and finally get: $$Var(T)\geq-log(\theta)\theta^2$$


(b) $$L(x,\lambda)=\prod_{i=1}^{n}{\frac{e^{-\lambda}\lambda^{x_i}}{x_i!}}=\frac{1}{\prod{x_i!}}e^{-n\lambda}\lambda^{\sum{x_i}}=\frac{1}{\prod{x_i!}}exp(-n\lambda)exp(log(\lambda)\sum{x_i})$$ So by denoting $\eta=log(\lambda), S(x)=\sum{x_i}, A(\eta)=n\lambda, h(x)=\frac{1}{\prod{x_i!}}$ we get a representation of the likelihood function as an exponential family.

Reparametrizing using $\theta$ we get $\eta=log(-log(\theta)), A(\eta)=-n log(\theta), S(x)=\sum{x_i}, h(x)=\frac{1}{\prod{x_i!}}$, again en exponential family. This is important as we get that $S(x)=\sum{x_i}$ is a sufficient statistic due to properties of exponential families (Fisher-Neyman factorization theorem).

$S(x)$ is the sum of $x_1,...,x_n\sim Pois(-log(\theta))$, so $S(x)$ itself is a Poisson RV with parameter $-nlog(\theta)$. Now, let $g(S(x))$ be a function s.t $E[g(S)]=0$:

$$E[g(S)]=\theta^{-n}\sum_{s=0}^{\infty}{g(s)\cdot\frac{\left(-n log(\theta) \right)^s}{s!}}$$ The only term that can be zero is $g(s)$, so if $E[g(S(x))]=0$ we conclude that $g(s)=0$ for all values of $s$, hence $S(x)=\sum{x_i}$ is a complete statistic. QED!


(c)

This is almost cheating: The corollary from the Lehmann-Scheffé theorem is that if S is a complete sufficient statistic, then applying the R-B procedure would yield the UMVUE.

last momentary pause: we were told "combine your findings in Parts (a) and (b)", so let's do it

In part (a) we got $Var(T)\geq-log(\theta)\theta^2$, in part (b) we got that $S(x)=\sum{x_i}$ is complete sufficient statistic, R-B has yielded $T^*$, so $Var(T^* )\geq-log(\theta)\theta^2$. That's it.

PS It would be proper to thank Prof. Pavel Chigansky of HUJI, who has taught us this exact problem in his statistical inference course.

$\endgroup$
2
  • $\begingroup$ @sirman canonical enough? $\endgroup$
    – Spätzle
    Jul 14, 2021 at 13:19
  • 1
    $\begingroup$ spelling of my name is wrong, anyway thanks! $\endgroup$
    – simran
    Jul 14, 2021 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.