# Use the Lehmann-Scheffé theorem to deduce that $\overline{X}$ is an UMVUE estimator for $\theta$

Let $$X_1,X_2,\ldots,X_n$$ be a random sample whose distribution is $$X\sim\operatorname{Bernoulli}(\theta)$$.

(a) Prove that $$\sum_{i=1}^n X_i$$ is complete.

(b) Use the Lehmann-Scheffé to deduce that $$\overline{X}$$ is an UMVUE estimator for $$\theta$$.

MY ATTEMPT

(a) Take two observations $$\textbf{y}$$ and $$\textbf{w}$$. Considering that $$p(\textbf{x}\mid\theta) = \theta^{\sum x_i}(1-\theta)^{n - \sum x_i}$$

we have $$p(\textbf{y}\mid\theta) = p(\textbf{w}\mid\theta) \Rightarrow p(\textbf{y} \mid \theta) = \theta^{\sum y_i}(1-\theta)^{n - \sum y_i} = p(\textbf{w}\mid\theta) = \theta^{\sum w_i}(1-\theta)^{n - \sum w_i}$$

from whence we get that $$\sum_{i=1}^n y_i = \sum_{i=n} w_i \Longrightarrow T(\textbf{y}) = T(\textbf{w})$$ where $$T(\textbf{x}) = \sum_{i=1}^n x_i$$ Thence we conclude that $$T$$ is complete.

Can someone take it from here and finish the exercise?

• Please add the self-study tag. This forum is not to be used for someone else to "finish the exercise". May 31, 2019 at 4:44
• Use the exponential family result to argue completeness of $T$ or prove it directly from definition. After that taking expectation of $T$ gives the answer immediately. May 31, 2019 at 5:21

To say that it is complete means its family of distributions is complete, and that means it admits no nontrivial unbiased estimators of zero. An unbiased estimator of zero is a function $$g$$ for which we have \begin{align} & \operatorname E (g(T(X_1,\ldots,X_n))\mid\theta) \\[8pt] = {} & \operatorname E(g(X_1+\cdots+X_n)\mid\theta) = 0 \text{ for ALL values of }\theta. \end{align} So we find the expectation: $$\operatorname E(g(X_1+\cdots+X_n)) = \sum_{x=0}^n g(x) \binom n x \theta^x (1-\theta)^{n-x}.$$ As a function of $$\theta$$ this is a polynomial of degree $$n.$$ Elementary algebra tells us that if a polynomial function is equal to $$0$$ at infinitely many points (i.e. in this case for ALL values of $$\theta$$) then all of its coefficients are $$0.$$ This will entail that $$g(x) = 0$$ for $$x=0,1,2,\ldots,n.$$ Thus an unbiased estimator of $$0$$ can only be $$0,$$ i.e. can only be the trivial estimator.

That proves completeness.

One must also prove that $$T= X_1+\cdots+X_n$$ is sufficient. That means the conditional distribution of $$X_1,\ldots,X_n$$ given $$T$$ does not depend on $$\theta.$$ We have \begin{align} & \Pr(X_1=x_1\ \&\ \cdots\ \&\ X_n=x_n\mid X_1+\cdots+X_n=t) \\[8pt] = {} & \frac{\Pr(X_1=x_1\ \&\ \cdots\ \&\ X_n=n\ \&\ X_1+\cdots+X_n=t)}{\Pr(X_1+\cdots+X_n=t)} \\[8pt] = {} & \frac{\Pr(X_1=x_1\ \&\ \cdots\ \&\ X_n=n)}{\Pr(X_1+\cdots+X_n=t)} = \frac{\theta^t (1-\theta)^{n-t}}{\binom n t \theta^t(1-\theta)^{n-t}} \\[8pt] = {} & 1\left/\binom n t\right. \end{align} and no $$\text{“}\theta\text{''}$$ appears in that last expression.

That proves sufficiency.

Now we need the fact that $$X_1$$ is an unbiased estimator of $$\theta$$ (albeit a very bad one for reasonable purposes, but that doesn't matter here).

Lehmann–Scheffe then tells us that the UMVUE is $$\operatorname E(X_1\mid T).$$ We have \begin{align} & \operatorname E(X_1\mid T=t) = \Pr(X_1=1\mid T=t) \\[8pt] = {} & \frac{\Pr(X_1=1\ \&\ \overbrace{X_1+\cdots+X_n}^\text{from 1 to n} = t)}{\Pr(T=t)} \\[8pt] = {} & \frac{\Pr(X_1=1\ \&\ \overbrace{X_2+\cdots+X_n}^\text{from 2 to n} = t-1)}{\Pr(T=t)} \\[8pt] = {} & \frac{\theta \cdot \binom{n-1}{t-1} \theta^{t-1} (1-\theta)^{n-t} }{\binom n t \theta^t (1-\theta)^{n-t}} = \frac{\binom{n-1}{t-1}}{\binom n t} \\[8pt] = {} & \frac{(n-1)!}{(t-1)!(n-t)!} \cdot \frac{t!(n-t)!}{n!} = \frac t n. \end{align}

Since $$\operatorname E(X_1\mid T=t) = t/n,$$ we conclude that $$\operatorname E(X_1\mid T) = T/n.$$

Thus $$T/n$$ is the UMVUE.