Suppose that we have $X_1,...,X_n$ iid observations from a Geometric($p$) distribution.

I found that the MLE of p is $\hat{p} = \frac{n}{n+\sum{X_i}}$.

I am trying to find the MLE of $\theta = \frac{p}{1-p}, \hat{\theta}$ and the Fisher Information of a single observation $I_1(\theta)$.

Now, by the functional invariance property of MLE estimators, I found that the MLE of $\hat{\theta}$ is just:

$\hat{\theta}$= $\frac{\hat{p}}{1-\hat{p}}$.

However, I have no idea how to find the Fisher Information $I_1(\theta)$.

I know of two ways of doing this, one is to exploit the asympotic efficiency and the lower bound princple of MLE's. That is, $Var(\hat{\theta})=\frac{1}{nI_1(\theta)}$.

Also, is the parametrization formula

${\mathcal I}_\eta(\eta) = {\mathcal I}_\theta(\theta(\eta)) \left( \frac{{\mathrm d} \theta}{{\mathrm d} \eta} \right)^2$ where ${\mathcal I}_\eta$ and ${\mathcal I}_\theta$ are the Fisher information measures of $η$ and $θ$, respectively.

Is there another easier way to do this? I have no idea how to find the variance above. Thank you!


1 Answer 1


By the formula for the MLE, I understand that you are dealing with the variant of the Geometric distribution where the random variables can take the value $0$. In this case we have

$$E(X_1) = \frac {1-p}{p},\,\,\, \text {Var}(X_1) = \frac {1-p}{p^2}$$

The Fisher Information of a single observation can be derived by applying its definition :

$$I_1(p) = \operatorname{E} \left[\left. \left(\frac{\partial}{\partial p} \ln f(X_1;p)\right)^2\right|p\right] = \operatorname{E} \left[\left. \left(\frac{\partial}{\partial p} \ln(1-p)^{X_1}p \right)^2\right|p \right]$$

$$=\operatorname{E} \left(-\frac {X_1}{1-p}+\frac 1p \right)^2 = \operatorname{E} \left(\frac {X_1^2}{(1-p)^2}+\frac 1{p^2}-2\frac {X_1}{(1-p)p}\right)$$

$$=\frac 1{p^2} - \frac {2}{(1-p)p} E(X_1)+ \frac {1}{(1-p)^2}\left(\text {Var}(X_1) + (E[X_1])^2\right)$$ $$=\frac 1{p^2}- \frac {2}{(1-p)p}\cdot \frac {1-p}{p} + \frac {1}{(1-p)^2}\left( \frac {1-p}{p^2} + \frac {(1-p)^2}{p^2}\right)$$

$$=\frac 1{p^2}- \frac {2}{p^2}+\frac {1}{(1-p)p^2}+\frac 1{p^2} = \frac {1}{(1-p)p^2}$$

We also have $$\frac {d\theta}{dp} = \frac {1}{(1-p)^2}$$ So $$I_1(\theta) = I_1(p)\cdot \left(\frac {d\theta}{dp} \right)^{-2} = \frac {1}{(1-p)p^2}\cdot (1-p)^4 = \frac {(1-p)^3}{p^2}$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.