I am reading this classic paper (Information and the Accuracy Attainable in the Estimation of Statistical Parameters) by CR Rao where he introduces the idea of minimizing the variance of an unbiased estimate using sufficient statistics. I have some difficulty in following his 2-line-proof - specifically Equations (3.7-3.8).

I will reproduce his proof here.

Say $T$ is the sufficient statistic, then the probability distribution $\phi(x ; \theta)$ can be written as $$ \phi(x ; \theta)=\Phi(T, \theta) \psi\left(x_{1}, \ldots, x_{n}\right) $$ where $\psi$ is independent of $\theta$.

If $t$ is an unbiased estimate, then $$ \theta=\int t \phi \pi d x_{i}=\int f(T) \Phi(T, \theta) d T $$ where $f(T)$ is a function independent of $\theta$

Shouldn't there be a Jacobian term due to change of variables from $x_i$ to $T$?

Further, $$ \begin{aligned} \int\left(t-\theta^{2}\right) \phi \pi d x_{i} &=\int[t-f(T)]^{2} \phi \pi d x_{i}+\int[f(T)-\theta]^{2} \Phi(T, \theta) d T \\ & \geq \int[f(T)-\theta]^{2} \Phi(T, \theta) d T \end{aligned} $$

How does the first equality follow?

Thanks in advance,

P.S.: I am familiar with the other ways of proving the Rao-Blackwellization theorem using variance decomposition lemma. I specifically want to understand the author's approach.


The first argument is that, if there exists an estimator $\delta(X_1,\ldots,X_n)=\delta(\mathbf X)$ of $\theta$ then \begin{align}\theta&=\Bbb E_\theta[\delta(\mathbf X)]\\ &=\Bbb E_\theta\{\underbrace{\Bbb E_\theta[\delta(\mathbf X)\mid T]}_{\text{call this }f(T)}\}\\ &=\Bbb E_\theta[f(T)]\\ &=\int f(t) \Phi(t;\theta)\text{d}t\end{align} assuming $\Phi(t;\theta)$ is the density of $T$

The second argument is Pythagore's theorem, namely that \begin{align} \text{var}(\delta) &=\int\overbrace{\left(\delta(\mathbf x)-\theta\right)^{2}}^{\text{square outside}} \phi(\mathbf x;\theta) \text{d}\mathbf x\\ &=\mathbb E_\theta[(\delta(\mathbf X)-\mathbb E[\delta(\mathbf X)|T(\mathbf X)]+\mathbb E[\delta(\mathbf X)|T(\mathbf X)]-\theta)^2]\\ &=\mathbb E_\theta[(\delta(\mathbf X)-\underbrace{\mathbb E[\delta(\mathbf X)|T(\mathbf X)]}_{\text{this is }f(T)})^2]+0+\mathbb E_\theta[(\underbrace{\mathbb E[\delta(\mathbf X)|T(\mathbf X)]}_{\text{this is }f(T)}-\theta)^2]\qquad\text{[Pythagore]}\\ &=\int[\delta(\mathbf x)-f(T(\mathbf x))]^{2} \phi(\mathbf x;\theta) \text{d}\mathbf x+\int[f(t)-\theta]^{2} \Phi(t, \theta) \text{d}t \end{align}


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.