Sufficiency in Lehmann Scheffe

Question

We are wondering what sufficiency in the Lehmann Scheffe Theorem is needed for. Our reasoning was:

1. If an unbiased estimator is uncorrelated with all unbiased estimators of 0, it is UMVUE
2. If the estimator is from a complete family, it is uncorrelated with all unbiased estimators of 0
3. Therefore an unbiased estimator from a complete family is UMVUE
4. UMVUE is unique

So what exactly is sufficiency needed for? Why is completeness not enough?

Would be nice if anyone could point out the flaws in our reasoning and explain!

Answer 1

The problem is (2), as others have noted. Let $X_1, ..., X_n$ be iid normal with mean $\theta$ and variance $1$. The statistic $T(X_1, ..., X_n) = X_1$ is complete, because the normal family is complete. But it is not uncorrelated with all unbiased estimators of $0$; take $\hat 0 = X_1 - X_2$. In constructing unbiased estimators of $0$ we are allowed to use the entire data, whereas completeness is only a property of the marginal distribution of $T(X_1, ..., X_n)$.

The logic in Casella and Berger is this: if $T = T(X_1, ..., X_n)$ is sufficient then it suffices to only consider the distribution of $T$ when looking for unbiased estimators, by Rao-Blackwell. This is what sufficiency is giving you - it allows you to ignore everything except $T$. So it suffices to show, if $g(T)$ is unbiased for $\theta$, that $g(T)$ is uncorrelated with every unbiased estimate of $0$, $\hat 0(T)$ [note: because of sufficiency, we have reduced the problem of showing uncorrelatedness with every estimator of $0$ to only have to show it for estimators that depend only on $T$]. But if $T$ is complete then there are no unbiased estimators $\hat 0(T)$ other than $0$ which $g(T)$ is uncorrelated with, so we are done. Of course, now that we've established that $g(T)$ is the UMVUE, it follows a posteriori that $g(T)$ is uncorrelated with all unbiased estimators of $0$, $\hat 0(X_1, ..., X_n)$, that depend on the entire sample.

Answer 2

If your Uniformly Minimum Variance Unbiased Estimator $U$ of $\theta$ were not a function of a sufficient statistic $S$ alone (a.s.), then, by the Rao-Blackwell Theorem, the random variable $V=\textrm{E}_\theta[U\mid S]$ would be an unbiased estimator of $\theta$ that has variance uniformly smaller than $U$, which contradicts the fact that $U$ is an Uniformly Minimum Variance Unbiased Estimator of $\theta$. A Lehmann-Scheffé style result without the sufficiency assumption would leave that window open.

Answer 3

Even though this is an old question I thought it would be helpful to have a more detailed answer.

Step 2 only follows for sure if your complete estimator $T$ is also sufficient. Suppose $U$ is any random variable with $E(U)=0$. Then $$ E(E(U|T)) = E(U) = 0 \Rightarrow E(U|T) = 0 $$ Note that this result depends on sufficiency, because $E(U|T)$ needs to be the same function for all $\theta$, so generally it can't depend on $\theta$. It follows that $$ Cov(T,U) = E(TU) -E(T)E(U) = E(TU) = E(TE(U|T)) = E(T*0) = 0. $$ Take the example by guy above. There we have $E(U|T) = E(X_1-X_2|X_1) = X_1-\theta$, a function that depends on $\theta$, and thus we don't necessarily get zero covariance with every mean zero random variable.