Cramer-Rao Lower Bound Proof (fuzzy step)

Question

The following is the derivation of the Cramer-Rao lower bound as detailed on p.336 of Casella and Berger's Statistical Inference:

$\frac{d}{d\theta}E[W(\bf{X})|\theta] = \int_{\chi}W(\bf{x})\left[\frac{\partial}{\partial\theta}f(\bf{x}|\theta)\right]dx=E\left[W(\bf{X})\cdot\frac{\partial}{\partial\theta}log\;f(\bf{x}|\theta)|\theta\right]$

the derivation then applies the condition $W(\bf{X}) = 1\quad$ therefore:

$0=\frac{d}{d\theta}E[1] =E\left[1\cdot\frac{\partial}{\partial\theta}log\;f(\bf{x}|\theta)|\theta\right]$

and

$Cov\left[W(\bf{X})\cdot\frac{\partial}{\partial\theta}log\;f(\bf{x}|\theta)|\theta\right]=\frac{d}{d\theta}E[W(\bf{X})|\theta]$

$Var\left[\frac{\partial}{\partial\theta}log\;f(\bf{x}|\theta)|\theta\right]=E\left[\left(\frac{\partial}{\partial\theta}log\;f(\bf{x}|\theta)\right)^2|\theta\right]$

Leading to the lower bound inequality:

$Var(W(\bf{X})|\theta) \ge \frac{(\frac{d}{d\theta}E[W(\bf{X})|\theta])^2}{E\left[\left(\frac{\partial}{\partial\theta}log\;f(\bf{x}|\theta)\right)^2|\theta\right]}$

(this arises directly from the Cauchy-Schwarz inequality where $Cov(X,Y)^2 \leq Var(X)Var(Y)$)

However it seems to me that this crucially depends on $W(\bf{X}) = 1$, (which would mean that if my estimator is not 1 then CRLB doesn't apply). I'm certain there's something I'm missing here and I'm hoping someone could help me out

Answer 1

The authors' use of $W(X)=1$ is admittedly a bit confusing. They don't want to say that the estimator has to be 1 for all samples of $X$ (which would be absurd). Rather, they want to emphasize that the expectation $E_\theta(\frac{\partial}{\partial \theta}\log f(X|\theta))$ can simply be calculated using the identity labelled (7.3.7) in their book.

What they are really doing here is proving the important result that, under certain regularity conditions, the expected value of the score function is always 0. To see this, write the expectation as an integral (or, for discrete values of $X$, a sum): \begin{align} E_\theta(\frac{\partial}{\partial \theta}\log f(X|\theta)) &= \int_X \frac{\partial}{\partial \theta}\log f(x|\theta) f(x|\theta) dx \\ &= \int_X \frac{\frac{\partial}{\partial \theta}f(x|\theta)}{f(x|\theta)}f(x|\theta)dx \\ &= \int_X \frac{\partial}{\partial \theta}f(x|\theta)dx. \end{align} If the regularity conditions are met, we can switch the order of integration and differentiation, giving \begin{align} \int_X \frac{\partial}{\partial \theta}f(x|\theta)dx = \frac{\partial}{\partial \theta} \int_X f(x|\theta)dx = \frac{\partial}{\partial \theta}1 = 0. \end{align} Note that here, we just used that by definition, the integral of a probability distribution over its support is 1, and that the derivative of any constant function is 0.

More on wikipedia: https://en.wikipedia.org/wiki/Score_(statistics)#Properties.