# Rao-Blackwell and unbiased estimators of zero

In Casella & Berger we are trying to prove that any estimator $$\phi$$ based on a complete sufficient statistic T is the unique best unbiased estimator of its expectation.

However, in the preceding example, where the "proof" is developed:

(Copied from a website that copied from Casella-Berger; in C&B, example 7.3.22)

The preceding theorem in C&B, 7.3.20, says:

$$W$$ is the best unbiased estimator of some $$\tau(\theta)$$ if and only if $$W$$ is uncorrelated with all unbiased estimators of 0

Note, this theorem does NOT refer to zero-unbiased estimators based on some sufficient statistics ^

I cannot figure, why do we need to consider only unbiased estimators of zero, which are based on the sufficient statistic $$Y$$ (statement in parentheses), instead of ALL 0-estimators?

And what does Rao-Blackwell have to do with this?

Rao-Blackwell shows that any estimator that is not based on a sufficient statistic cannot be made worse (and is often improved in practice) by conditioning it upon a sufficient statistic, i.e., converting it to one that is based on a sufficient statistic. If (and only if) the original estimator is unbiased, the Rao-Blackwellized estimator is also unbiased. It is therefore sufficient :) to consider only those estimators based on sufficient statistics when looking for a UMVUE, because every estimator that is not in that set has a corresponding estimator in the set that has the same or better performance.

• I think you answered a wrong question. It is not about the improvement of the target estimator for e.g. $\theta$. The preceding theorem in C&B, 7.3.20, says that "any estimator is the best unbiased if and only if it is uncorrelated with ALL unbiased estimators of ZERO". This is what the comment in parenthesis refers to. Why does the author say we need to consider ONLY UNBIASED ZERO ESTIMATORS BASED ON SUFF.STAT, instead of ALL ZERO-UNBIASED ESTIMATORS. I will modify the question. Commented Sep 15, 2021 at 15:50
• That question is exactly answered by my response. There are no zero-unbiased estimators that are better than the best zero-unbiased estimator based on a sufficient statistic, by the Rao-Blackwell theorem. Why look at estimators that are guaranteed not to be better than the best? Commented Sep 15, 2021 at 17:40
• we should look at them, because the theorem assumes it? The theorem (I updated the question) says that the target sufficient statistic should be uncorrelated with ALL unbiased zero estimators to be 'best unbiased' (for some parameter NOT zero), not only those conditioned on the sufficient statistic. Commented Sep 15, 2021 at 18:21
• Also, the proof of this theorem ('estimator best unbiased iff it is uncorrelated with all unbiased zero estimators' above) does not in any way use Rao-Blackwell theorem. Commented Sep 15, 2021 at 18:23
• Your questions are: "I cannot figure, why do we need to consider only unbiased estimators of zero, which are based on the sufficient statistic Y (statement in parentheses), instead of ALL 0-estimators? And what does Rao-Blackwell have to do with this?". My answer addresses both questions. The first answer is: you only need to consider unbiased estimators based on sufficient statistics because there aren't any unbiased estimators not based on sufficient statistics that are better. The second one is: we know this because of Rao-Blackwell, which is what Rao-Blackwell has to do with it. Commented Sep 15, 2021 at 19:26

It appears that the poster is already aware that the best unbiased estimator will have to be a statistic that will be a function of a sufficient statistic.

Let $$U(Y)$$ be an unbiased estimator of 0 based on $$Y$$. In the example the authors want to show $$Y$$ is the best unbiased estimator of $$\theta$$. They do this by arguing that $$E(U(Y))=0$$ implies $$U(Y)=0$$ a.s (since Y is a complete statistic) and hence $$Cov(Y,U(Y))=0$$. The poster is wondering: why are the authors only considering unbiased estimators of 0 like $$U(Y)$$ when Theorem 7.3.20 states that an unbiased estimator is best iff it has zero covariance with ALL unbiased estimators of 0?.

First possible explanation:

Let W be some arbitrary unbiased estimator of $$\theta$$. If $$U$$ is an unbiased estimator of 0 then $$W+aU$$ is also an unbiased estimator of $$\theta$$. Now let $$\phi(Y)=E(W+aU|Y)$$ where Y is a sufficient statistic (also complete). By the Rao-Blackwell theorem $$Var(\phi(Y)) \leq Var(W+aU)$$. Note that $$\phi(Y)=E(W|Y)+aE(U|Y)$$. Letting $$U(Y)=E(U|Y)$$, we can conclude that we should only look at unbiased estimators of 0 based on $$Y$$.

Second possible explanation:

Let $$\phi(T)$$ be an unbiased estimator of $$\theta$$ where $$T$$ is a sufficient statistic. Now let's look at $$\phi(T)+aU$$. This addition either yields an estimator based entirely on T or not. If it does not, then we should ignore $$\phi(T)+aU$$ as a possible candidate for best unbiased estimator. If $$\phi(T)+aU$$ yields an estimator based on $$T$$ only say $$\phi^*(T)$$, then clearly $$U$$ must be an estimator based on $$T$$ alone ie $$U=\frac{\phi^*(T)-\phi(T)}{a}$$. Hence we need to only focus on unbiased estimators of 0 based on $$T$$

I was also struggled with this statement pointed by the poster: by sufficiency, in the form of the Rao-Blackwell Theorem, we need consider only unbiased estimators of zero based on Y where Y is a complete and sufficient in the context. The authors didn't give much detail explaining why we should only check if the proposed unbiased estimator is uncorrelated with unbiased estimators only based on a complete and sufficient statistic. After some search, I find that an unbiased estimator uncorrelated with unbiased estimator of zero based on sufficient and complete statistic implies the unbiased estimator uncorrelated with all unbiased estimator of zero.

For example, let $$h(\tilde{T})$$ be an unbiased estimator of $$g(\theta)$$ based on $$\tilde{T}$$ which is complete and sufficient. Let $$U$$ be an unbiased estimator of zero. $$U_t$$ be unbiased estimator of zero based on $$\tilde{T}$$. Then the following shows what I claimed above：

Let $$\phi_t$$=$$h(\tilde{T})+cU$$ where c is a constant. Here we have $$\phi_t$$ is an unbiased estimator of zero because:

E($$\phi_t$$)=E($$h(\tilde{T})+cU$$)=E($$h(\tilde{T})$$)+cE($$U$$)=$$g(\theta)$$+0=$$g(\theta)$$

Now we want to know if $$h(\tilde{T})$$ can be updated by adding random noise (in this case $$U$$). Since both $$h(\tilde{T})$$ and $$\phi_t$$ are unbiased estimators, to decide which one is better, we compare their corresponding variances:

Var($$\phi_t$$)= Var($$h(\tilde{T})+cU$$)=Var($$h(\tilde{T})$$)+$$c^2$$Var($$U$$)+2$$c$$Cov($$\tilde{T},U$$)

This implies Var($$\phi_t$$) $$\geq$$ Var($$h(\tilde{T})$$) unless Cov($$h(\tilde{T}),U$$)=0

• Note: Cov($$h(\tilde{T}),U$$) can be negative so that Var($$\phi_t$$) $$\le$$ Var($$h(\tilde{T})$$)
• Cov($$h(\tilde{T}),U$$) =E($$h(\tilde{T})U$$) because E($$U$$)=0

Here $$U$$ is all unbiased estimator of zero. The poster asks why $$h(\tilde{T})$$ uncorrelated with $$U_t$$ is sufficient in applying theorem 7.3.20. I hope the following answers the questions:

Cov($$h(\tilde{T}),U$$) =E($$h(\tilde{T})U$$) = E{E[$$h(\tilde{T})U|\tilde{T}$$]}=E{$$h(\tilde{T})E[U|\tilde{T}]$$}=Cov($$h(\tilde{T}),E[U|\tilde{T}]$$)

where $$E[U|\tilde{T}]$$ is unbiased estimator of zero based on $$\tilde{T}$$

So $$h(\tilde{T})$$ and $$E[U|\tilde{T}]$$ uncorrelated implies $$h(\tilde{T})$$ and $$U$$ uncorrelated.

This is my very first post on StackExchange. Please comment if I made any mistake.