7
$\begingroup$

Suppose $X_1, X_2, . . . , X_n$ are i.i.d Poisson ($\theta$) random variables, where $\theta\in(0,\infty)$. Give the UMVUE of $\theta e^{-\theta}$

I found a similar problem here.

I have that the Poisson distribution is an exponential family where

$$f(x\mid\theta)=\frac{\theta^x e^{-\theta}}{x!}=\left(\frac{I_{(0,1,...,)}(x)}{x!}\right)e^{-\theta}\text{exp}\left(x\cdot\text{log}(\theta)\right)$$

where $w(\theta)=\text{log}(\theta)$ and $t(x)=x$. Since

$$\{{w(\theta):\theta\in\Theta\}}=\{{\text{log}(\theta):\theta\in(0,\infty)\}}=(-\infty,\infty)$$

contains an open set in $\mathbb{R}$

$$T(\vec{X})=\sum_{i=1}^n t(X_i)=\sum_{i=1}^n X_i$$

is a complete statistic (Complete Statistics in the Exponential Family Theorem) and is also sufficient (Factorization Theorem). Hence $\bar{X}=\frac{T}{n}$ is sufficient for $\theta$, and hence, for $\theta e^{-\theta}$.

I tried to Rao-Blackwellize the unbiased estimator $\bar{X}$. For all possible values of $T$ we have that since $\theta e^{-\theta}=\mathsf P(X_i=1)$ then

$$\begin{align*} \mathsf P(X_1=1\mid\bar{X}=t) &=\frac{\mathsf P(X_1=1,\sum_{i=2}^n {X_i} =nt-1)}{\mathsf P(\bar{X} = t)}\\\\ &=\frac{\frac{e^{-\theta}\theta}{1}\cdot\frac{e^{-(n-1)\theta}((n-1)\theta)^{nt-1}}{nt-1!}}{\frac{e^{-n\theta}(n\theta)^{nt}}{nt!}}\\\\ &=t\cdot\left(1-\frac{1}{n}\right)^{nt-1} \end{align*}$$

Since $\mathsf E(X_1)=\theta$, then $X_1$ is an unbiased estimator of $\theta$, and so it follows from the Lehmann–Scheffé theorem that $t\cdot\left(1-\frac{1}{n}\right)^{nt-1}$ is the UMVUE.

Is this a valid solution? Were my justifications correct and sufficient?

$\endgroup$
  • 1
    $\begingroup$ It looks right to me - and the form is also approximately of the form $T(x)e^{T(X)-1}$ since $(1-1/n)^n \to e$, which is what we would intuitively expect. $\endgroup$ – Xiaomi Oct 25 '18 at 0:58
  • 1
    $\begingroup$ This answer seems to confirm your working $\endgroup$ – Xiaomi Oct 25 '18 at 1:00
  • $\begingroup$ It's been a long day and I should really be watching mindless TV, so maybe I'm missing something, but I think you may have an error. If $\lambda = 5$ and $n = 20,$ then I get an improbable result. Please check. $\endgroup$ – BruceET Oct 25 '18 at 2:13
4
$\begingroup$

The Poisson distribution is a one-parameter exponential family distribution, with natural sufficient statistic given by the sample total $T(\mathbf{x}) = \sum_{i=1}^n x_i$. The canonical form is:

$$p(\mathbf{x}|\theta) = \exp \Big( \ln (\theta) T(\mathbf{x}) - n\theta \Big) \cdot h(\mathbf{x}) \quad \quad \quad h(\mathbf{x}) = \coprod_{i=1}^n x_i! $$

From this form it is easy to establish that $T$ is a complete sufficient statistic for the parameter $\theta$. So the Lehmann–Scheffé theorem means that for any $g(\theta)$ there is only one unbiased estimator of this quantity that is a function of $T$, and this is the is UMVUE of $g(\theta)$. One way to find this estimator (the method you are using) is via the Rao-Blackwell theorem --- start with an arbitrary unbiased estimator of $g(\theta)$ and then condition on the complete sufficient statistic to get the unique unbiased estimator that is a function of $T$.

Using Rao-Blackwell to find the UMVUE: In your case you want to find the UMVUE of:

$$g(\theta) \equiv \theta \exp (-\theta).$$

Using the initial estimator $\hat{g}_*(\mathbf{X}) \equiv \mathbb{I}(X_1=1)$ you can confirm that,

$$\mathbb{E}(\hat{g}_*(\mathbf{X})) = \mathbb{E}(\mathbb{I}(X_1=1)) = \mathbb{P}(X_1=1) = \theta \exp(-\theta) = g(\theta),$$

so this is indeed an unbiased estimator. Hence, the unique UMVUE obtained from the Rao-Blackwell technique is:

$$\begin{equation} \begin{aligned} \hat{g}(\mathbf{X}) &\equiv \mathbb{E}(\mathbb{I}(X_1=1) | T(\mathbf{X}) = t) \\[6pt] &= \mathbb{P}(X_1=1 | T(\mathbf{X}) = t) \\[6pt] &= \mathbb{P} \Big( X_1=1 \Big| \sum_{i=1}^n X_i = t \Big) \\[6pt] &= \frac{\mathbb{P} \Big( X_1=1 \Big) \mathbb{P} \Big( \sum_{i=2}^n X_i = t-1 \Big)}{\mathbb{P} \Big( \sum_{i=1}^n X_i = t \Big)} \\[6pt] &= \frac{\text{Pois}(1| \theta) \cdot \text{Pois}(t-1| (n-1)\theta)}{\text{Pois}(t| n\theta)} \\[6pt] &= \frac{t!}{(t-1)!} \cdot \frac{ \theta \exp(-\theta) \cdot ((n-1) \theta)^{t-1} \exp(-(n-1)\theta)}{(n \theta)^t \exp(-n\theta)} \\[6pt] &= t \cdot \frac{ (n-1)^{t-1}}{n^t} \\[6pt] &= \frac{t}{n} \Big( 1- \frac{1}{n} \Big)^{t-1} \\[6pt] \end{aligned} \end{equation}$$

Your answer has a slight error where you have conflated the sample mean and the sample total, but most of your working is correct. As $n \rightarrow \infty$ we have $(1-\tfrac{1}{n})^n \rightarrow \exp(-1)$ and $t/n \rightarrow \theta$, so taking these asymptotic results together we can also confirm consistency of the estimator:

$$\hat{g}(\mathbf{X}) = \frac{t}{n} \Big[ \Big( 1- \frac{1}{n} \Big)^n \Big] ^{\frac{t}{n} - \frac{1}{n}} \rightarrow \theta [ \exp (-1) ]^\theta = \theta \exp (-\theta) = g(\theta).$$

This latter demonstration is heuristic, but it gives a nice check on the working. It is interesting here that you get an estimator that is a finite approximation to the exponential function of interest.

$\endgroup$
  • 1
    $\begingroup$ Thanks for clarification. Should I delete simulation? $\endgroup$ – BruceET Oct 25 '18 at 2:59
  • 1
    $\begingroup$ I thinks your simulation is pretty cool (+1) - a nice way of showing the OP that his estimator is biased. I would keep it if I were you. In general, it is nice to have a theory answer juxtaposed with a simulation answer. $\endgroup$ – Reinstate Monica Oct 25 '18 at 3:02
  • $\begingroup$ Thank you Ben. I will accept your answer once I fully understand it. Is $\mathbb{I}$ your notation for the indicator function? $\endgroup$ – Remy Oct 25 '18 at 3:34
  • $\begingroup$ @Remy: Yes, that is correct. $\endgroup$ – Reinstate Monica Oct 25 '18 at 4:00
2
$\begingroup$

Here is a simulation in R that I did using a the average of $n = 20$ observations, where $\lambda = 5.$ The parameter par is $P(X = 1) = \lambda e^{-\lambda}.$ The estimate par.fcn, which tries to estimate $P(X = 1)$ merely as a function of the average, is biased. My version of your UMVUE for $P(X=1)$ using a function of average a (instead of total) seems to work OK.

set.seed(2018);  m = 10^5; n = 20; lam=5; par=dpois(1, lam)
x = rpois(m*n, lam);  MAT=matrix(x, nrow=m)  # each row a sample of size 20
a = rowMeans(MAT)

lam.umvue = a;  lam;  mean(lam.umvue);  sd(lam.umvue)
[1] 5             # exact lambda
[1] 5.000788      # mean est of lambda
[1] 0.4989791     # aprx SD of est

par.fcn = exp(-lam.umvue)*lam.umvue;  par;  mean(par.fcn);  sd(par.fcn)
[1] 0.03368973    # exact P(X=1)
[1] 0.03620296    # slightly biased
[1] 0.01444379
sqrt(mean((par.fcn - par)^2))
[1] 0.01466074    # aprx root mean square error (rmse) of par.fun

par.umvue = a*(1-1/n)^(n*a - 1);  par;  mean(par.umvue);   sd(par.umvue)
[1] 0.03368973    # exact P(X=1)
[1] 0.03365454    # mean est of P(X=1), seems unbiased
[1] 0.01388531
sqrt(mean((par.umvue - par)^2))
[1] 0.01388528    # aprx rmse of umvue of P(X=1);  smaller than rmse of par.fun
$\endgroup$
  • $\begingroup$ Im not sure I follow your answer, though admittedly I am unfamilar with this language. If $a$ is the sample mean, then $a (1-1/n)^{(n*a-1)}$ is the UMVUE he derived, is it not? His derived UMVUE is $\bar{X}(1-1/n)^{n \bar{X}-1}$ $\endgroup$ – Xiaomi Oct 25 '18 at 2:47
  • $\begingroup$ I took t to be $\sum_i X_i,$ not $\bar X.$ $\endgroup$ – BruceET Oct 25 '18 at 2:50
  • $\begingroup$ I see. He conditioned on $\bar{X}$ so here $t$ is the sample mean, not total.I can see the confusion though since he defines the sum as $T$ first to justify using the sample mean $\endgroup$ – Xiaomi Oct 25 '18 at 2:53
  • $\begingroup$ At the very least there is a confusion of definition. Note displayed $T(\vec{X})=\sum_{i=1}^n t(X_i)=\sum_{i=1}^n X_i.$ But I think you're right about the conditioning. $\endgroup$ – BruceET Oct 25 '18 at 2:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.