Let $X_1,X_2,\ldots,X_n$ be a random sample whose distribution is $X\sim\operatorname{Bernoulli}(\theta)$.

(a) Prove that $\sum_{i=1}^n X_i$ is complete.

(b) Use the Lehmann-Scheffé to deduce that $\overline{X}$ is an UMVUE estimator for $\theta$.


(a) Take two observations $\textbf{y}$ and $\textbf{w}$. Considering that $$ p(\textbf{x}\mid\theta) = \theta^{\sum x_i}(1-\theta)^{n - \sum x_i} $$

we have $$ p(\textbf{y}\mid\theta) = p(\textbf{w}\mid\theta) \Rightarrow p(\textbf{y} \mid \theta) = \theta^{\sum y_i}(1-\theta)^{n - \sum y_i} = p(\textbf{w}\mid\theta) = \theta^{\sum w_i}(1-\theta)^{n - \sum w_i} $$

from whence we get that $$ \sum_{i=1}^n y_i = \sum_{i=n} w_i \Longrightarrow T(\textbf{y}) = T(\textbf{w}) $$ where $$ T(\textbf{x}) = \sum_{i=1}^n x_i $$ Thence we conclude that $T$ is complete.

Can someone take it from here and finish the exercise?

  • 1
    $\begingroup$ Please add the self-study tag. This forum is not to be used for someone else to "finish the exercise". $\endgroup$ – Xi'an May 31 '19 at 4:44
  • $\begingroup$ Use the exponential family result to argue completeness of $T$ or prove it directly from definition. After that taking expectation of $T$ gives the answer immediately. $\endgroup$ – StubbornAtom May 31 '19 at 5:21

To say that it is complete means its family of distributions is complete, and that means it admits no nontrivial unbiased estimators of zero. An unbiased estimator of zero is a function $g$ for which we have \begin{align} & \operatorname E (g(T(X_1,\ldots,X_n))\mid\theta) \\[8pt] = {} & \operatorname E(g(X_1+\cdots+X_n)\mid\theta) = 0 \text{ for ALL values of }\theta. \end{align} So we find the expectation: $$ \operatorname E(g(X_1+\cdots+X_n)) = \sum_{x=0}^n g(x) \binom n x \theta^x (1-\theta)^{n-x}. $$ As a function of $\theta$ this is a polynomial of degree $n.$ Elementary algebra tells us that if a polynomial function is equal to $0$ at infinitely many points (i.e. in this case for ALL values of $\theta$) then all of its coefficients are $0.$ This will entail that $g(x) = 0$ for $x=0,1,2,\ldots,n.$ Thus an unbiased estimator of $0$ can only be $0,$ i.e. can only be the trivial estimator.

That proves completeness.

One must also prove that $T= X_1+\cdots+X_n$ is sufficient. That means the conditional distribution of $X_1,\ldots,X_n$ given $T$ does not depend on $\theta.$ We have \begin{align} & \Pr(X_1=x_1\ \&\ \cdots\ \&\ X_n=x_n\mid X_1+\cdots+X_n=t) \\[8pt] = {} & \frac{\Pr(X_1=x_1\ \&\ \cdots\ \&\ X_n=n\ \&\ X_1+\cdots+X_n=t)}{\Pr(X_1+\cdots+X_n=t)} \\[8pt] = {} & \frac{\Pr(X_1=x_1\ \&\ \cdots\ \&\ X_n=n)}{\Pr(X_1+\cdots+X_n=t)} = \frac{\theta^t (1-\theta)^{n-t}}{\binom n t \theta^t(1-\theta)^{n-t}} \\[8pt] = {} & 1\left/\binom n t\right. \end{align} and no $\text{“}\theta\text{''}$ appears in that last expression.

That proves sufficiency.

Now we need the fact that $X_1$ is an unbiased estimator of $\theta$ (albeit a very bad one for reasonable purposes, but that doesn't matter here).

Lehmann–Scheffe then tells us that the UMVUE is $\operatorname E(X_1\mid T).$ We have \begin{align} & \operatorname E(X_1\mid T=t) = \Pr(X_1=1\mid T=t) \\[8pt] = {} & \frac{\Pr(X_1=1\ \&\ \overbrace{X_1+\cdots+X_n}^\text{from $1$ to $n$} = t)}{\Pr(T=t)} \\[8pt] = {} & \frac{\Pr(X_1=1\ \&\ \overbrace{X_2+\cdots+X_n}^\text{from $2$ to $n$} = t-1)}{\Pr(T=t)} \\[8pt] = {} & \frac{\theta \cdot \binom{n-1}{t-1} \theta^{t-1} (1-\theta)^{n-t} }{\binom n t \theta^t (1-\theta)^{n-t}} = \frac{\binom{n-1}{t-1}}{\binom n t} \\[8pt] = {} & \frac{(n-1)!}{(t-1)!(n-t)!} \cdot \frac{t!(n-t)!}{n!} = \frac t n. \end{align}

Since $\operatorname E(X_1\mid T=t) = t/n,$ we conclude that $\operatorname E(X_1\mid T) = T/n.$

Thus $T/n$ is the UMVUE.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.