# Sufficient Statistic and Unbiased Estimate in Exponential Family

I am reading this classic paper (Information and the Accuracy Attainable in the Estimation of Statistical Parameters) by CR Rao where he deals with sufficient statistics in exponential distributions building on Koopman's work (eqs 3.11-3.12).

An exponential distribution $$\phi(x,\theta)$$ is given by $$\phi(x, \theta)=\exp \left(\Theta_{1} X_{1}+\Theta_{2}+X_{2}\right)$$ where $$X_1$$ and $$X_2$$ are functions of $$x_1,\dots,x_n$$ and $$\Theta_1$$ and $$\Theta_2$$ are functions of $$\theta$$

We can derive the expecatation $$E(X_1)$$ and variance $$Var(X_1)$$ easily as follows \begin{align} \int \exp \left(\Theta_{1} X_{1}+\Theta_{2}+X_{2}\right) & d x_{i}=1 \; \; \text{(differentiating on both sides) }\\ \int X_1 \exp \left(\Theta_{1} X_{1}+\Theta_{2}+X_{2}\right) d x_{i} d \Theta_1 + & \int \exp \left(\Theta_{1} X_{1}+\Theta_{2}+X_{2}\right) d x_{i} d \Theta_2 = 0 \\ \implies E\left(X_{1}\right)&=-\frac{d \Theta_{2}}{d \Theta_{1}} \\ \int X_1 \exp \left(\Theta_{1} X_{1}+\Theta_{2}+X_{2}\right) d x_{i} & = -\frac{d \Theta_{2}}{d \Theta_{1}} \; \; \text{(differentiating on both sides again) }\\ \int X_1^2 \exp \left(\Theta_{1} X_{1}+\Theta_{2}+X_{2}\right) d x_{i} \frac{d \Theta_{1}}{d \theta} + & \int X_1 \exp \left(\Theta_{1} X_{1}+\Theta_{2}+X_{2}\right) d x_{i} \frac{d \Theta_{2}} {d \theta} \\ =& -\frac{d}{d \theta} (\frac{d \Theta_{2}}{d \Theta_{1}}) \\ E(X^2) - [E(X)]^2 = - \frac{\frac{d}{d \theta} (\frac{d \Theta_{2}}{d \Theta_{1}})}{\frac{d \Theta_1}{d \theta}} & = - \frac{d^2 \Theta_2}{d \Theta_1^2} \end{align}

The author claims that $$X_1$$ is the best unbiased estimate of $$-\frac{d \Theta_{2}}{d \Theta_{1}}$$ and proves it as follows:

$$\left\{\frac{d}{d \theta} \frac{d \Theta_{2}}{d \Theta_{1}}\right\}^{2} /\left\{\frac{d^{2} \Theta_{2}}{d \Theta_{1}^{2}} \frac{d \Theta_{1}}{d \theta}\right\}=-\frac{d^{2} \Theta_{2}}{d \Theta_{1}^{2}}=V\left(X_{1}\right) \; \; \qquad\text{(1)}$$

I am able to follows this step only partially. We know that the generalized CRLB is given as $$V(t) \ge \left\{f^{\prime}(\theta)\right\}^{2} / I$$ where $$t$$ is the unbiased estimate of some function $$f(\theta)$$ instead of $$\theta$$. Therefore, the numerator $$\left\{\frac{d}{d \theta} \frac{d \Theta_{2}}{d \Theta_{1}}\right\}^{2}$$ follows. I am not sure how the denominator and further simplification follows in equation (1). Possible Errata?

There is a typo in the paper. The denominator should be $$-\left\{\frac{d^{2} \Theta_{2}}{d \Theta_{1}^{2}} \left( \frac{d \Theta_{1}}{d \theta} \right)^2\right\}$$.
\begin{align} I & = \Bbb E\left( \left( \frac{d (\Theta_1X_1 + \Theta_2) }{d \theta} \right)^2 \right) \\ & = \Bbb E \left( X^2 ( \frac{d \Theta_1}{ d \theta})^2 + 2X \frac{d \Theta_1}{ d \theta} \frac{d \Theta_2}{ d \theta} + (\frac{d \Theta_2}{ d \theta})^2 \right) \\ & \qquad \text{Substituting the expectation.}\\ & = \Bbb E \left( X^2 ( \frac{d \Theta_1}{ d \theta})^2 - (\frac{d \Theta_2}{ d \theta})^2 \right) \\ & \qquad \text{Substituting the second moment.} \\ & = \left( ( \frac{d^2 \Theta_1}{ d \Theta_1^2}+ ( \frac{d \Theta_2}{ d \Theta_1})^2) ( \frac{d \Theta_1}{ d \theta})^2 - (\frac{d \Theta_2}{ d \theta})^2 \right) \\ & = - \frac{d^{2} \Theta_{2}}{d \Theta_{1}^{2}} \left( \frac{d \Theta_{1}}{d \theta} \right)^2 \end{align}