Why does the Rao-Blackwell Theorem require $\Bbb E(\hat{\theta}^2) < \infty$?

Question

The Rao-Blackwell Theorem states

Let $\hat{\theta}$ be an estimator of $\theta$ with $\Bbb E (\hat{\theta}^2) < \infty$ for all $\theta$. Suppose that $T$ is sufficient for $\theta$, and let $\theta ^ * = \Bbb E (\hat{\theta}|T)$ Then for all $\theta$, $$\Bbb E (\theta^* - \theta )^2 \leq \Bbb E (\hat{\theta} - \theta )^2$$ The inequality is strict unless $\hat{\theta}$ is a function of $T$

If I understand this theorem correctly, this states that, if I have a sufficient statistic $T$ for $\theta$, then the conditional expected value of $\hat{\theta}$ given $T$ is the solution to $\min_{\hat{\theta}} \Bbb E $$(\hat{\theta}-\theta)^2$

My Quesitons

1. Am I correct that $\theta^*$ minimizes $\Bbb E $$(\hat{\theta}-\theta)^2$ ?
2. Why does the Rao-Blackwell Theorem require $\Bbb E(\hat{\theta}^2) < \infty$?
3. Why is the inequality strict unless $\hat{\theta}$ is a function of $T$ ?

Answer 1

1. No, $\theta^*$ is a better estimator than $\hat\theta$ but not necessarily the best (whatever that means!)
2. If the estimator has no variance, then its risk is infinite and there is no guarantee that $\theta^*$ has a finite risk (even though this may happen as pointed out by Horst Grünbusch in his comments).
3. Under finite variance for $\hat\theta$, the inequality is strict because of the variance decomposition as the sum of the expected conditional variance plus the variance of the conditional expectation $$\text{var}(\hat\theta)=\mathbb{E}_T[\text{var}(\hat\theta|T)]+ \text{var}_T(\mathbb{E}[\hat\theta|T])=\mathbb{E}_T[\text{var}(\theta|T)]+\text{var}_T(\theta^*)$$ Unless the expected conditional variance is zero, which amounts to $\hat\theta$ a function of $T$ only.

Answer 2

1. Note that being a sufficient statistic is not unique. Trivially, the whole data are sufficient, but conditioning an estimator on them doesn't change anything. So a sufficient statistic alone is not sufficient (pun!) for having minimal mean squared error. See the Lehmann-Scheffé-theorem, which uses the Rao-Blackwell-theorem in the proof, for a sufficient sufficiency (in fact, being sufficient and complete).

2. If both are infinite, the weak inequality is always true. But then, as a counterexample, you can construct a sufficient statistic that is not a function of $T$ but has still infinite variance (such that only $\leq$ holds).

Take for example $C_1 \sim t_2 + \mu$, a shifted $t_2$-distributed random variable with $E(C_1) = \mu$ and $Var(C_1) = \infty$, and as another independent random variable $C_2 \sim t_2$. The parameter to estimate is $\mu$. Original estimator is $\hat{\theta} = C_1 + C_2$. A sufficient statistic is of course $C_1$. Both the Rao-Blackwell estimator $E(\hat{\theta}|C_1)=C_1$ and $\hat{\theta}$ have infinite variance. So the inequality would hold weakly. On the other hand, $C_1+C_2$ is not a mere function of $C_1$: It involves the other random variable, so that would be a contradiction to the last sentence you asked your 3rd question about. In fact, some textbooks admit infinite variance for the original estimator, but in turn they cannot state when $<$ holds.

3. If $\hat{\theta}$ is a function of $T$, you can prove by the factorization theorem that $\hat{\theta}$ is already sufficient for $\theta$. So again we end up with improving nothing. Apart from this case, the inequality is strict, and that's the non-trivial assertion of the theorem.