7
$\begingroup$

We are wondering what sufficiency in the Lehmann Scheffe Theorem is needed for. Our reasoning was:

  1. If an unbiased estimator is uncorrelated with all unbiased estimators of 0, it is UMVUE
  2. If the estimator is from a complete family, it is uncorrelated with all unbiased estimators of 0
  3. Therefore an unbiased estimator from a complete family is UMVUE
  4. UMVUE is unique

So what exactly is sufficiency needed for? Why is completeness not enough?

Would be nice if anyone could point out the flaws in our reasoning and explain!

$\endgroup$
4
  • 3
    $\begingroup$ How did you arrive at your (2)? Can you think of an example of an unbiased statistic that is complete but not sufficient? (Hint: Bernoulli.) What does sufficiency buy you, particularly when evaluating quadratic risk of unbiased estimators? $\endgroup$
    – cardinal
    Sep 20, 2012 at 19:01
  • $\begingroup$ Our reasoning for (2) was: If we are in a complete family, all unbiased estimators of 0 are 0 with probability 1. Every statistic is uncorrelated with 0, so in particular our estimator is. Our problem might be, that the "we are in a complete family" is a little vague. We only assume that the estimator is from a complete family. Can there be another estimator from a non-complete family then? We have no idea! $\endgroup$ Sep 20, 2012 at 19:39
  • 1
    $\begingroup$ It seems you are confused about the notions of unbiasedness and also probably completeness. If a statistic is complete with respect to a parameter $\theta$ that does not mean it is uncorrelated with every unbiased statistic of zero (of the whole data). Try constructing a counterexample from a sample of $\mathrm{Ber}(p)$. Going through this exercise should solidify the concepts, I think. $\endgroup$
    – cardinal
    Sep 20, 2012 at 19:55
  • $\begingroup$ It seems like we are using different definitions of completeness. What do you mean by a statistic being complete for a parameter? To us, only a family of distributions can be complete (not for a parameter). In Casella&Berger it says: If a family of pdfs or pmfs f(x|theta) has the property that there are no unbiased estimators of 0 (other than 0 itself) ... Recall that the property of completeness guarantees such a situation. (p. 346) Maybe you can clarify for us, which family should be complete here. Of the statistic? Of the sample? Or how is it meant? Thanks! $\endgroup$ Sep 20, 2012 at 20:27

3 Answers 3

10
$\begingroup$

The problem is (2), as others have noted. Let $X_1, ..., X_n$ be iid normal with mean $\theta$ and variance $1$. The statistic $T(X_1, ..., X_n) = X_1$ is complete, because the normal family is complete. But it is not uncorrelated with all unbiased estimators of $0$; take $\hat 0 = X_1 - X_2$. In constructing unbiased estimators of $0$ we are allowed to use the entire data, whereas completeness is only a property of the marginal distribution of $T(X_1, ..., X_n)$.

The logic in Casella and Berger is this: if $T = T(X_1, ..., X_n)$ is sufficient then it suffices to only consider the distribution of $T$ when looking for unbiased estimators, by Rao-Blackwell. This is what sufficiency is giving you - it allows you to ignore everything except $T$. So it suffices to show, if $g(T)$ is unbiased for $\theta$, that $g(T)$ is uncorrelated with every unbiased estimate of $0$, $\hat 0(T)$ [note: because of sufficiency, we have reduced the problem of showing uncorrelatedness with every estimator of $0$ to only have to show it for estimators that depend only on $T$]. But if $T$ is complete then there are no unbiased estimators $\hat 0(T)$ other than $0$ which $g(T)$ is uncorrelated with, so we are done. Of course, now that we've established that $g(T)$ is the UMVUE, it follows a posteriori that $g(T)$ is uncorrelated with all unbiased estimators of $0$, $\hat 0(X_1, ..., X_n)$, that depend on the entire sample.

$\endgroup$
1
  • 2
    $\begingroup$ Returning to this question a few years later, I'm not entirely satisfied with my answer. Another idea is that (1) given an unbiased estimator, I can always do at least as well using one which depends only on $T$ (Rao-Blackwell) while (2) by completeness, there exists only one unbiased estimator based on $T$ (by completeness and the fact that if I had two unbiased estimators I could construct an unbiased estimator of $0$ other than $0$). $\endgroup$
    – guy
    Jul 1, 2016 at 15:10
3
$\begingroup$

If your Uniformly Minimum Variance Unbiased Estimator $U$ of $\theta$ were not a function of a sufficient statistic $S$ alone (a.s.), then, by the Rao-Blackwell Theorem, the random variable $V=\textrm{E}_\theta[U\mid S]$ would be an unbiased estimator of $\theta$ that has variance uniformly smaller than $U$, which contradicts the fact that $U$ is an Uniformly Minimum Variance Unbiased Estimator of $\theta$. A Lehmann-Scheffé style result without the sufficiency assumption would leave that window open.

$\endgroup$
3
  • $\begingroup$ We agree that there would exist a contradiction with Rao Blackwell, if you had Lehmann Scheffe without sufficiency. Still we are not sure what exactly is wrong with our reasoning. $\endgroup$ Sep 20, 2012 at 19:42
  • 1
    $\begingroup$ cardinal already told you that the flaw was at step 2. Maybe you are misinterpreting Casella and Berger. What you wrote was a rather incomplete paraphrase. $\endgroup$ Sep 20, 2012 at 20:45
  • $\begingroup$ Yes certainly we misinterpret Casella & Berger, but we don't know how exactly. After proving step (3) (Thm. 7.3.20) they argue that if we have a complete family we'd be "done". (because 0 would be the only unbiased estimator of 0). So we interpreted that as (2). How would you interpret it? Thanks a lot! $\endgroup$ Sep 20, 2012 at 20:56
1
$\begingroup$

Even though this is an old question I thought it would be helpful to have a more detailed answer.

Step 2 only follows for sure if your complete estimator $T$ is also sufficient. Suppose $U$ is any random variable with $E(U)=0$. Then $$ E(E(U|T)) = E(U) = 0 \Rightarrow E(U|T) = 0 $$ Note that this result depends on sufficiency, because $E(U|T)$ needs to be the same function for all $\theta$, so generally it can't depend on $\theta$. It follows that $$ Cov(T,U) = E(TU) -E(T)E(U) = E(TU) = E(TE(U|T)) = E(T*0) = 0. $$ Take the example by guy above. There we have $E(U|T) = E(X_1-X_2|X_1) = X_1-\theta$, a function that depends on $\theta$, and thus we don't necessarily get zero covariance with every mean zero random variable.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.