# Trying to make sense of claims regarding Rao-Blackwell and Lehmann-Scheffé for sufficient/complete statistics

I am currently trying to learn the two related concepts of the Rao-Blackwell theorem and the Lehmann-Scheffé theorem.

Assume we have the random sample $$X_1, \dots, X_n$$ with mean $$\mu$$ and variance $$\sigma^2 < \infty$$. We have that $$E[S^2] = \sigma^2$$, where $$S^2 = \sum_{i = 1}^n \dfrac{(X_i - \bar{X})^2}{n - 1}$$ and $$\bar{X} = \sum_{i = 1}^n \dfrac{X_i}{n}$$. Now assume the $$X_i$$ are Poisson random variables with parameter $$\lambda$$. My understanding is that, using Lehmann-Scheffé, we get that $$E[S^2 \mid \bar{X}] = \bar{X}$$. Then, using the law of total variance, we get that $$\text{Var}(S^2) > \text{Var}(\bar{X})$$. Based on what I've read, the two above theorems imply that, if we define, say, a sufficient statistic $$T_1(\mathbf{X})$$ and a complete sufficient statistic $$T_2(\mathbf{X})$$ for some parameter $$\varphi$$, then, under some conditions, we can say that $$\text{Var}(T_1(\mathbf{X})) > \text{Var}(T_2(\mathbf{X}))$$. However, I'm having trouble understanding this last part. I've read over various notes on the subject, but I'm still not sure I understand what it's saying. Why can we say that $$\text{Var}(T_1(\mathbf{X})) > \text{Var}(T_2(\mathbf{X}))$$? And what are these 'conditions' that make this inequality valid?

• $E[S^2|\bar{X}] = \bar{X}$??? Commented Apr 14, 2021 at 17:10
• I can't think of an example of an incomplete and sufficient statistic and another complete and sufficient statistic for the same probability model. R-B says if you take an unbiased statistic and condition on the sufficient statistic, you will get a new estimator with lower variance but there may be other unbiased estimators with even lower variance. L-S says if it's complete and sufficient, you get the UMVUE. Commented Apr 14, 2021 at 17:15
• I think the Hodge's Superefficient estimator is a simple counter example that actually helps understand how ill-formed some of this early variance bound research was: en.wikipedia.org/wiki/Hodges%27_estimator Commented Apr 14, 2021 at 17:21
• @AdamO Sorry about that. I've been trying to read from multiple documents explaining the same thing, so I got confused when typing out my question. Is that better? Commented Apr 14, 2021 at 17:31
• @SextusEmpiricus Oh my goodness, you're right. It seems that I completely screwed it up. The problem is that this was all based on memory, so I was trying to recall the idea and then ask. And it seems that, in doing so, I confused myself and wrote the wrong thing! My apologies! Given what I have written, it seems that what Thomas wrote is absolutely correct. Again, I apologise for any confusion. Commented Apr 22, 2021 at 5:58

There is a complete sufficient statistic for $$\theta$$ in a model $${\cal P}_\theta$$ if and only if the minimal sufficient statistic is complete (according to Lehmann "An Interpretation of Completeness and Basu’s Theorem"). This means you can't have distinct $$T_1(X)$$ and $$T_2(X)$$ the way you want. As the paper says (first complete paragraph of the second column)

On the other hand, existence of a complete sufficient statistic is equivalent to the completeness of the minimal sufficient statistic, and hence is a property of the model $${\cal P}$$.

That is, in any given model, if $$T_2$$ is complete sufficient and $$T_1$$ is sufficient, $$T_1$$ is also complete sufficient.

The two theorems say

1/ Rao-Blackwell: Conditioning on any sufficient statistic will reduce the variance. This follows from the law of total variance

2/ Lehmann-Scheffé: In the special case that the model has a complete sufficient statistic, you get a fully efficient estimator.

In the Poisson case, the minimal sufficient statistic $$\bar X$$ is complete and the two estimators are identical.

There's an interesting example here of a situation where a Rao-Blackwell-type estimator is not fully efficient (not even admissible). The model is $$X\sim U[\theta(1-k),\theta(1+k)]$$ for known $$k$$ and unknown $$\theta$$. The Cramér-Rao bound does not apply, since the range of $$X$$ depends on $$\theta$$. A sufficient statistic is the pair $$(\min X_1, \max X_n)$$, and any observation is an unbiased estimator, however $$E[X_1|(\min X_1, \max X_n)]$$ is not even the best linear function of the two components of the sufficient statistic.

A couple more points to fill in potential gaps:

1. This leaves open is whether there might be an unbiased estimator attaining the Cramér-Rao bound that isn't obtainable using the Lehmann-Scheffé theorem. There isn't (in reasonably nice models): any model where the bound is attained has a score function of the form $$\frac{\partial \ell}{\partial \theta}=I(\theta)(g(x)-\theta)$$ for some $$g()$$ (where $$I()$$ is the information), in which case $$g(x)$$ is a both a complete sufficient statistic for $$\theta$$ and the minimum variance unbiased estimator.

2. As @AdamO indicates, none of this translates tidily to asymptotics: there are asymptotically unbiased estimators that beat the asymptotic information bound at a point (Hodges superefficient estimator) and even on a dense set of measure zero(Le Cam's extension of Hodge's estimator). The best you can do is the Local Asymptotic Minimax theorem, which says you can't beat an 'efficient' estimator uniformly over neighbourhoods of $$\theta_0$$ with diameter $$O(n^{-1/2})$$

• Thanks again, Thomas. Are you sure about what you said in the first part? Isn't $E[S^2 \mid \bar{X}] = \bar{X}$ the special case of Lehmann-Scheffé? I was told that, if we have two statistics, $T_2(\mathbf{X})$ and $T_1(\mathbf{X})$ are a complete sufficient statistic and statistic, respectively, for some parameter of interest $\theta$, then $\text{Var}(T_1(\mathbf{X})) > \text{Var}(T_2(\mathbf{X}))$. So my question is then as follows: under what conditions does this inequality hold? Commented Apr 19, 2021 at 18:49
• The point is it never holds for the same $\theta$. If $T_2$ is complete sufficient and $T_1$ is sufficient, then $T_1$ will also be complete. Commented Apr 19, 2021 at 21:51
• Hmm, that's very interesting. Is there are theorem that says this, or something that I can point to to justify this reasoning ("this never holds for the same $\theta$, because ...")? Commented Apr 19, 2021 at 21:53
• The first link is to a paper by Lehmann that says the minimal sufficient statistic is complete in any model with a complete sufficient statistic Commented Apr 20, 2021 at 1:15
• This is a great answer, thanks! I was very confused as to how to know when a Rao-Blackwell-type estimator is an MVUE; I see now that completeness is the key. Are there any results on when statistics obtained from factorization are complete? Commented Jun 21, 2021 at 21:57