I am reading this classic paper (Information and the Accuracy Attainable in the Estimation of Statistical Parameters) by CR Rao where he deals with sufficient statistics in exponential distributions building on Koopman's work (eqs 3.11-3.12).

An exponential distribution $\phi(x,\theta)$ is given by $$ \phi(x, \theta)=\exp \left(\Theta_{1} X_{1}+\Theta_{2}+X_{2}\right) $$ where $X_1$ and $X_2$ are functions of $x_1,\dots,x_n$ and $\Theta_1$ and $\Theta_2$ are functions of $\theta$

We can derive the expecatation $E(X_1)$ and variance $Var(X_1)$ easily as follows \begin{align} \int \exp \left(\Theta_{1} X_{1}+\Theta_{2}+X_{2}\right) & d x_{i}=1 \; \; \text{(differentiating on both sides) }\\ \int X_1 \exp \left(\Theta_{1} X_{1}+\Theta_{2}+X_{2}\right) d x_{i} d \Theta_1 + & \int \exp \left(\Theta_{1} X_{1}+\Theta_{2}+X_{2}\right) d x_{i} d \Theta_2 = 0 \\ \implies E\left(X_{1}\right)&=-\frac{d \Theta_{2}}{d \Theta_{1}} \\ \int X_1 \exp \left(\Theta_{1} X_{1}+\Theta_{2}+X_{2}\right) d x_{i} & = -\frac{d \Theta_{2}}{d \Theta_{1}} \; \; \text{(differentiating on both sides again) }\\ \int X_1^2 \exp \left(\Theta_{1} X_{1}+\Theta_{2}+X_{2}\right) d x_{i} \frac{d \Theta_{1}}{d \theta} + & \int X_1 \exp \left(\Theta_{1} X_{1}+\Theta_{2}+X_{2}\right) d x_{i} \frac{d \Theta_{2}} {d \theta} \\ =& -\frac{d}{d \theta} (\frac{d \Theta_{2}}{d \Theta_{1}}) \\ E(X^2) - [E(X)]^2 = - \frac{\frac{d}{d \theta} (\frac{d \Theta_{2}}{d \Theta_{1}})}{\frac{d \Theta_1}{d \theta}} & = - \frac{d^2 \Theta_2}{d \Theta_1^2} \end{align}

The author claims that $X_1$ is the best unbiased estimate of $-\frac{d \Theta_{2}}{d \Theta_{1}}$ and proves it as follows:

$$ \left\{\frac{d}{d \theta} \frac{d \Theta_{2}}{d \Theta_{1}}\right\}^{2} /\left\{\frac{d^{2} \Theta_{2}}{d \Theta_{1}^{2}} \frac{d \Theta_{1}}{d \theta}\right\}=-\frac{d^{2} \Theta_{2}}{d \Theta_{1}^{2}}=V\left(X_{1}\right) \; \; \qquad\text{(1)} $$

I am able to follows this step only partially. We know that the generalized CRLB is given as $$ V(t) \ge \left\{f^{\prime}(\theta)\right\}^{2} / I $$ where $t$ is the unbiased estimate of some function $f(\theta)$ instead of $\theta$. Therefore, the numerator $\left\{\frac{d}{d \theta} \frac{d \Theta_{2}}{d \Theta_{1}}\right\}^{2}$ follows. I am not sure how the denominator and further simplification follows in equation (1). Possible Errata?


1 Answer 1


There is a typo in the paper. The denominator should be $-\left\{\frac{d^{2} \Theta_{2}}{d \Theta_{1}^{2}} \left( \frac{d \Theta_{1}}{d \theta} \right)^2\right\}$.

It is obtained as follows:

\begin{align} I & = \Bbb E\left( \left( \frac{d (\Theta_1X_1 + \Theta_2) }{d \theta} \right)^2 \right) \\ & = \Bbb E \left( X^2 ( \frac{d \Theta_1}{ d \theta})^2 + 2X \frac{d \Theta_1}{ d \theta} \frac{d \Theta_2}{ d \theta} + (\frac{d \Theta_2}{ d \theta})^2 \right) \\ & \qquad \text{Substituting the expectation.}\\ & = \Bbb E \left( X^2 ( \frac{d \Theta_1}{ d \theta})^2 - (\frac{d \Theta_2}{ d \theta})^2 \right) \\ & \qquad \text{Substituting the second moment.} \\ & = \left( ( \frac{d^2 \Theta_1}{ d \Theta_1^2}+ ( \frac{d \Theta_2}{ d \Theta_1})^2) ( \frac{d \Theta_1}{ d \theta})^2 - (\frac{d \Theta_2}{ d \theta})^2 \right) \\ & = - \frac{d^{2} \Theta_{2}}{d \Theta_{1}^{2}} \left( \frac{d \Theta_{1}}{d \theta} \right)^2 \end{align}


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.