1
$\begingroup$

I am currently studying the Cramer-Rao lower bound. My notes say the following:

Theorem: Cramer-Rao lower bound
Let $Y_1, \dots, Y_n$ have a joint distribution $f_\theta (\mathbf{y})$, where $f_\theta (\mathbf{y})$ satisfies the following two regularity conditions:

  1. The support supp$f_\theta (\mathbf{y})$ does not depend on $\theta$;
  2. for any statistic $T = T(Y_1, \dots, Y_n)$ with a finite $E_\theta \lvert T \rvert$, integration w.r.t. $\mathbf{y}$ and differentiation w.r.t. $\theta$ in calculating $(E[T])_\theta^\prime$ can be interchanged, that is, $$(E[T])_\theta^\prime = \dfrac{d}{d\theta} \int T(\mathbf{y})f_\theta (\mathbf{y}) \ d\mathbf{y} = \int T(\mathbf{y})f_\theta^\prime (\mathbf{y}) \ d\mathbf{y}$$ Let $T$ be an unbiased estimator for $\theta$, with a finite variance. Then $$\text{Var}(T) \ge \dfrac{1}{I(\theta)},$$ where $I(\theta) = E[(\ln(f_\theta(\mathbf{Y})_\theta^\prime)^2]$ is called the Fisher information number.
    More generally, if $T$ is an unbiased estimator for $g(\theta)$, where $g(\cdot)$ is differentiable, then, $$\text{Var}(T) \ge \dfrac{(g^\prime(\theta))^2}{I(\theta)}$$
    Proof: Cramer-Rao lower bound
    Since $T$ is an unbiased estimator for $g(\theta)$, we have $$g(\theta) = ET = \int T(\mathbf{y}) f_\theta(\mathbf{y}) \ d\mathbf{y},$$ and, under regularity conditions, $$g^\prime(\theta) = (E[T])_\theta^\prime = \dfrac{d}{d \theta} \int T(\mathbf{y}) f_\theta(\mathbf{y}) \ d\mathbf{y} = \int T(\mathbf{y}) f_\theta^\prime (\mathbf{y}) \ d\mathbf{y} = \int T(\mathbf{y})(\ln \left[ f_\theta (\mathbf{y})\right]_\theta^\prime f_\theta (\mathbf{y}) \ d\mathbf{y} = E[T(\mathbf{Y})(\ln [f_\theta(\mathbf{Y})])_\theta^\prime ]$$

I am confused by this part:

Since $T$ is an unbiased estimator for $g(\theta)$, we have $$g(\theta) = ET = \int T(\mathbf{y}) f_\theta(\mathbf{y}) \ d\mathbf{y}$$

Why does $T$ being an unbiased estimator for $g(\theta)$ imply that $g(\theta) = ET = \int T(\mathbf{y}) f_\theta(\mathbf{y}) \ d\mathbf{y}$?


EDIT

From https://en.wikipedia.org/wiki/Bias_of_an_estimator#Definition :

"... or equivalently, if the expected value of the estimator matches that of the parameter." In this case, the expected value of the estimator does not match the parameter, but rather a function of the parameter. So there seems to be some disconnect here between what's in these notes and the Wikipedia definition of an unbiased estimator.

$\endgroup$
3
  • 1
    $\begingroup$ What definition(s) and formula(s) do you know for expectation? $\endgroup$
    – whuber
    Jul 19 at 13:38
  • 3
    $\begingroup$ Re the edit: you misinterpret Wikipedia. What is being estimated here is $g(\theta).$ A fortiori, the estimator is unbiased if its expectation equals $g(\theta).$ Indeed, when $g$ is a one-to-one function, $g(\theta)$ is a parameter. $\endgroup$
    – whuber
    Jul 19 at 14:52
  • 1
    $\begingroup$ @whuber Ok, that makes sense. I think I was confused because $\theta$ itself is usually the parameter being estimated, but, in this case, the "parameter" is $g(\theta)$. $\endgroup$ Jul 19 at 14:57
3
$\begingroup$

This is the definition of unbiasedness.

$\endgroup$
3
  • $\begingroup$ Care to elaborate? en.wikipedia.org/wiki/Bias_of_an_estimator#Definition $\endgroup$ Jul 19 at 13:22
  • 1
    $\begingroup$ The Wiki already mentioned, that being unbiased means the expected value of the estimator being equal to the true (unknown) parameter value. Maybe you read the text wrong? It is $T$ (a statistic), not $Y$ (the raw data), that is the unbiased estimator. $\endgroup$ Jul 19 at 13:34
  • $\begingroup$ See my edit. "... or equivalently, if the expected value of the estimator matches that of the parameter." In this case, the expected value of the estimator does not match the parameter, but rather a function of the parameter. So there seems to be some disconnect here between what's in these notes and the Wikipedia definition of an unbiased estimator. $\endgroup$ Jul 19 at 14:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.