Rao-Blackwell and unbiased estimators of zero

Question

In Casella & Berger we are trying to prove that any estimator $\phi$ based on a complete sufficient statistic T is the unique best unbiased estimator of its expectation.

However, in the preceding example, where the "proof" is developed:

(Copied from a website that copied from Casella-Berger; in C&B, example 7.3.22)

The preceding theorem in C&B, 7.3.20, says:

$W$ is the best unbiased estimator of some $\tau(\theta)$ if and only if $W$ is uncorrelated with all unbiased estimators of 0

Note, this theorem does NOT refer to zero-unbiased estimators based on some sufficient statistics ^

I cannot figure, why do we need to consider only unbiased estimators of zero, which are based on the sufficient statistic $Y$ (statement in parentheses), instead of ALL 0-estimators?

And what does Rao-Blackwell have to do with this?

Answer 1

Rao-Blackwell shows that any estimator that is not based on a sufficient statistic cannot be made worse (and is often improved in practice) by conditioning it upon a sufficient statistic, i.e., converting it to one that is based on a sufficient statistic. If (and only if) the original estimator is unbiased, the Rao-Blackwellized estimator is also unbiased. It is therefore sufficient :) to consider only those estimators based on sufficient statistics when looking for a UMVUE, because every estimator that is not in that set has a corresponding estimator in the set that has the same or better performance.

Answer 2

It appears that the poster is already aware that the best unbiased estimator will have to be a statistic that will be a function of a sufficient statistic.

Let $U(Y)$ be an unbiased estimator of 0 based on $Y$. In the example the authors want to show $Y$ is the best unbiased estimator of $\theta$. They do this by arguing that $E(U(Y))=0$ implies $U(Y)=0$ a.s (since Y is a complete statistic) and hence $Cov(Y,U(Y))=0$. The poster is wondering: why are the authors only considering unbiased estimators of 0 like $U(Y)$ when Theorem 7.3.20 states that an unbiased estimator is best iff it has zero covariance with ALL unbiased estimators of 0?.

First possible explanation:

Let W be some arbitrary unbiased estimator of $\theta$. If $U$ is an unbiased estimator of 0 then $W+aU$ is also an unbiased estimator of $\theta$. Now let $\phi(Y)=E(W+aU|Y)$ where Y is a sufficient statistic (also complete). By the Rao-Blackwell theorem $Var(\phi(Y)) \leq Var(W+aU)$. Note that $\phi(Y)=E(W|Y)+aE(U|Y)$. Letting $U(Y)=E(U|Y)$, we can conclude that we should only look at unbiased estimators of 0 based on $Y$.

Second possible explanation:

Let $\phi(T)$ be an unbiased estimator of $\theta$ where $T$ is a sufficient statistic. Now let's look at $\phi(T)+aU$. This addition either yields an estimator based entirely on T or not. If it does not, then we should ignore $\phi(T)+aU$ as a possible candidate for best unbiased estimator. If $\phi(T)+aU$ yields an estimator based on $T$ only say $\phi^*(T)$, then clearly $U$ must be an estimator based on $T$ alone ie $U=\frac{\phi^*(T)-\phi(T)}{a}$. Hence we need to only focus on unbiased estimators of 0 based on $T$

Answer 3

I was also struggled with this statement pointed by the poster: by sufficiency, in the form of the Rao-Blackwell Theorem, we need consider only unbiased estimators of zero based on Y where Y is a complete and sufficient in the context. The authors didn't give much detail explaining why we should only check if the proposed unbiased estimator is uncorrelated with unbiased estimators only based on a complete and sufficient statistic. After some search, I find that an unbiased estimator uncorrelated with unbiased estimator of zero based on sufficient and complete statistic implies the unbiased estimator uncorrelated with all unbiased estimator of zero.

For example, let $h(\tilde{T})$ be an unbiased estimator of $g(\theta)$ based on $\tilde{T}$ which is complete and sufficient. Let $U$ be an unbiased estimator of zero. $U_t$ be unbiased estimator of zero based on $\tilde{T}$. Then the following shows what I claimed above：

Let $\phi_t$=$h(\tilde{T})+cU$ where c is a constant. Here we have $\phi_t$ is an unbiased estimator of zero because:

E($\phi_t$)=E($h(\tilde{T})+cU$)=E($h(\tilde{T})$)+cE($U$)=$g(\theta)$+0=$g(\theta)$

Now we want to know if $h(\tilde{T})$ can be updated by adding random noise (in this case $U$). Since both $h(\tilde{T})$ and $\phi_t$ are unbiased estimators, to decide which one is better, we compare their corresponding variances:

Var($\phi_t$)= Var($h(\tilde{T})+cU$)=Var($h(\tilde{T})$)+$c^2$Var($U$)+2$c$Cov($\tilde{T},U$)

This implies Var($\phi_t$) $\geq$ Var($h(\tilde{T})$) unless Cov($h(\tilde{T}),U$)=0

• Note: Cov($h(\tilde{T}),U$) can be negative so that Var($\phi_t$) $\le$ Var($h(\tilde{T})$)
• Cov($h(\tilde{T}),U$) =E($h(\tilde{T})U$) because E($U$)=0

Here $U$ is all unbiased estimator of zero. The poster asks why $h(\tilde{T})$ uncorrelated with $U_t$ is sufficient in applying theorem 7.3.20. I hope the following answers the questions:

Cov($h(\tilde{T}),U$) =E($h(\tilde{T})U$) = E{E[$h(\tilde{T})U|\tilde{T}$]}=E{$h(\tilde{T})E[U|\tilde{T}]$}=Cov($h(\tilde{T}),E[U|\tilde{T}]$)

where $E[U|\tilde{T}]$ is unbiased estimator of zero based on $\tilde{T}$

So $h(\tilde{T})$ and $E[U|\tilde{T}]$ uncorrelated implies $h(\tilde{T})$ and $U$ uncorrelated.

This is my very first post on StackExchange. Please comment if I made any mistake.