# Why does the MSE version of Rao-Blackwell theorem require $T$ to be a sufficient statistic?

The proof for the MSE version appears not to depend on $T$ being a sufficient statistic. I provide a minimal version here:

Let $\hat\theta,T$ be observable random variables, and let $\theta \in \mathbb R$ be fixed.

Define $\theta^+ \colon T \to \mathbb R$ with $\theta^+(t) = E[\hat\theta|t]$ or short $\theta^+ = E[\hat\theta| T]$.

Then we can proof that the mean-squared-error of $\theta^+$ wrt $\theta$ is less or equal to that of $\hat\theta$:

\begin{align} E_T[(\theta^+ - \theta)^2] &= E_T[(E[\hat\theta | T] - \theta)^2] \\ &= E_T[E[\hat\theta - \theta | T]^2] \\ &\leq E_T[E[(\hat\theta - \theta)^2 | T]] \\ &= E[(\hat\theta - \theta)^2] \\ \end{align}

I have marked expectations over $T$ instead of $\hat\theta$ as $E_T$. The justifications are

• first equality: definition of $\theta^+$
• second equality: linearity of expectation
• inequality: follows from definition on variance, or Jensen's inequality
• last equality: totality of expectation

So this proofs that a conditional random variable $\theta^+$ has less variance than the original random variable $\hat\theta$, over which it forms an expectation over some domain $T$. This is not very surprising, but apparently this is the heart of the MSE version of the RB theorem. I don't know about the version that uses risks, maybe the sufficiency is required there.

Am I missing something?

If $T$ is not sufficient, then the conditional expectation $E_\theta[\hat\theta(X) \mid T]$ may depend on $\theta$ (in the sense that you need to know $\theta$ in order to calculate it). So, in other words, your estimator is not a statistic (a function of the data) unless $T$ is sufficient.
To make this clear, suppose $T$ is just a constant, or otherwise trivial. Then for an unbiased estimator, we would have $E_\theta[\hat\theta(X) \mid T] = E_\theta[\hat\theta(X)] = \theta$. This is an very good estimator in theory, but obviously can't be used since it depends on the unknown quantity $\theta$.
Sufficiency is exactly the condition required to prevent this, since it means that the distribution will not depend on the unknown $\theta$. These lecture notes explain this more carefully.
• Hmm, I defined $\theta$ to be some constant. Apparently this fixes the problem you mention. But it prevents us from talking about estimators. Is this correct? But I'll think about this, and see how to accomodate. Commented Sep 1, 2017 at 11:37
• But wouldn't it be enough to say that $T$ is a statistic (without sufficiency)? As sufficiency appears to only require more information (i.e. all relevant information for estimating the parameter), and does not forbid too much information. Commented Sep 1, 2017 at 12:29
• It's standard for $\theta$ to be fixed in the sense of 'not random', but it has to be possible to vary it, otherwise estimation is pointless. You need sufficiency because this means that, once $T$ is known, the distribution of $X$ (and hence any function of it such as $\hat{\theta}(X)$) is independent of $\theta$. Commented Sep 3, 2017 at 15:22
• Your intuition about what happens when you add more information to $T$ isn't right: if $T=X$, for example, then $E_\theta[\hat{\theta}(X) | T] = \hat{\theta}(X)$, so it changes nothing. On the other hand, if $T$ is trivial, then $E_\theta[\hat{\theta}(X)] = \theta$, which we clearly can't use as an estimator! Commented Sep 3, 2017 at 15:25
• Ok, now I get what you mean. But, the equation in my question holds, although the expectation cannot be computed from the observations? But the sufficiency allows us to call the conditional expectation an estimator (which requires it to be a statistic, which requires it to be a function of the data). But this leaves me a bit puzzled. What if $\theta$ isn't the only hidden parameter of the distribution? Why must the conditional expectation be a function of the data? I find the argument in the lecture slides not compelling. Commented Sep 3, 2017 at 19:58