Use the Lehmann-Scheffé theorem to deduce that $\overline{X}$ is an UMVUE estimator for $\theta$

Question

Let $X_1,X_2,\ldots,X_n$ be a random sample whose distribution is $X\sim\operatorname{Bernoulli}(\theta)$.

(a) Prove that $\sum_{i=1}^n X_i$ is complete.

(b) Use the Lehmann-Scheffé to deduce that $\overline{X}$ is an UMVUE estimator for $\theta$.

MY ATTEMPT

(a) Take two observations $\textbf{y}$ and $\textbf{w}$. Considering that $$ p(\textbf{x}\mid\theta) = \theta^{\sum x_i}(1-\theta)^{n - \sum x_i} $$

we have $$ p(\textbf{y}\mid\theta) = p(\textbf{w}\mid\theta) \Rightarrow p(\textbf{y} \mid \theta) = \theta^{\sum y_i}(1-\theta)^{n - \sum y_i} = p(\textbf{w}\mid\theta) = \theta^{\sum w_i}(1-\theta)^{n - \sum w_i} $$

from whence we get that $$ \sum_{i=1}^n y_i = \sum_{i=n} w_i \Longrightarrow T(\textbf{y}) = T(\textbf{w}) $$ where $$ T(\textbf{x}) = \sum_{i=1}^n x_i $$ Thence we conclude that $T$ is complete.

Can someone take it from here and finish the exercise?

Answer 1

To say that it is complete means its family of distributions is complete, and that means it admits no nontrivial unbiased estimators of zero. An unbiased estimator of zero is a function $g$ for which we have \begin{align} & \operatorname E (g(T(X_1,\ldots,X_n))\mid\theta) \\[8pt] = {} & \operatorname E(g(X_1+\cdots+X_n)\mid\theta) = 0 \text{ for ALL values of }\theta. \end{align} So we find the expectation: $$ \operatorname E(g(X_1+\cdots+X_n)) = \sum_{x=0}^n g(x) \binom n x \theta^x (1-\theta)^{n-x}. $$ As a function of $\theta$ this is a polynomial of degree $n.$ Elementary algebra tells us that if a polynomial function is equal to $0$ at infinitely many points (i.e. in this case for ALL values of $\theta$) then all of its coefficients are $0.$ This will entail that $g(x) = 0$ for $x=0,1,2,\ldots,n.$ Thus an unbiased estimator of $0$ can only be $0,$ i.e. can only be the trivial estimator.

That proves completeness.

One must also prove that $T= X_1+\cdots+X_n$ is sufficient. That means the conditional distribution of $X_1,\ldots,X_n$ given $T$ does not depend on $\theta.$ We have \begin{align} & \Pr(X_1=x_1\ \&\ \cdots\ \&\ X_n=x_n\mid X_1+\cdots+X_n=t) \\[8pt] = {} & \frac{\Pr(X_1=x_1\ \&\ \cdots\ \&\ X_n=n\ \&\ X_1+\cdots+X_n=t)}{\Pr(X_1+\cdots+X_n=t)} \\[8pt] = {} & \frac{\Pr(X_1=x_1\ \&\ \cdots\ \&\ X_n=n)}{\Pr(X_1+\cdots+X_n=t)} = \frac{\theta^t (1-\theta)^{n-t}}{\binom n t \theta^t(1-\theta)^{n-t}} \\[8pt] = {} & 1\left/\binom n t\right. \end{align} and no $\text{“}\theta\text{''}$ appears in that last expression.

That proves sufficiency.

Now we need the fact that $X_1$ is an unbiased estimator of $\theta$ (albeit a very bad one for reasonable purposes, but that doesn't matter here).

Lehmann–Scheffe then tells us that the UMVUE is $\operatorname E(X_1\mid T).$ We have \begin{align} & \operatorname E(X_1\mid T=t) = \Pr(X_1=1\mid T=t) \\[8pt] = {} & \frac{\Pr(X_1=1\ \&\ \overbrace{X_1+\cdots+X_n}^\text{from $1$ to $n$} = t)}{\Pr(T=t)} \\[8pt] = {} & \frac{\Pr(X_1=1\ \&\ \overbrace{X_2+\cdots+X_n}^\text{from $2$ to $n$} = t-1)}{\Pr(T=t)} \\[8pt] = {} & \frac{\theta \cdot \binom{n-1}{t-1} \theta^{t-1} (1-\theta)^{n-t} }{\binom n t \theta^t (1-\theta)^{n-t}} = \frac{\binom{n-1}{t-1}}{\binom n t} \\[8pt] = {} & \frac{(n-1)!}{(t-1)!(n-t)!} \cdot \frac{t!(n-t)!}{n!} = \frac t n. \end{align}

Since $\operatorname E(X_1\mid T=t) = t/n,$ we conclude that $\operatorname E(X_1\mid T) = T/n.$

Thus $T/n$ is the UMVUE.