I have a random sample of size n that are distributed as Bernoulli random variables with parameter $p$. Given

$$v=\frac{p}{1-p}$$ we are asked to find the the MLE of $v$, say $\hat{v}$. Further, we are told that an unbiased estimator of $v$ does not exist, so using $\hat{v}$, we are asked to find the bias of $v$. Here's what I've done so far:

Note that $$v=\frac{p}{1-p}=\frac{1}{1-p} -1$$ $$\implies v+1=\frac{1}{1-p}$$ $$\implies 1-\frac{1}{1+v}=p$$ $$\implies p=\frac{v}{1+v}$$ $$\implies 1-p=\frac{1}{1+v}$$ Now, let $L(p|\boldsymbol{x})=\Pi_{i=1}^n f_i(x_i|p)$ denote the likelihood function of $p$. Then have $$L(p|\boldsymbol{x})=p^t(1-p)^{n-t}$$ $$\implies L(p|\boldsymbol{x})=(\frac{v}{1+v})^t(\frac{1}{1+v})^{n-t}$$ $$\implies L(p|\boldsymbol{x})=v^t(\frac{1}{1+v})^n$$ where $t=\sum X_i$. Now let $log(L(p|\boldsymbol{x}))=l(p|\boldsymbol{x})$. Then we have $$l(p|\boldsymbol{x})=t\ln(v) - n\ln(1+v)$$ $$\implies \frac{d\;l(p|\boldsymbol{x})}{dv}=\frac{t}{v}-\frac{n}{1+v}$$ Setting the derivative equal to 0, we obtain $$\frac{t}{v}-\frac{n}{1+v}=0$$ $$\implies t(1+v) - nv = 0$$ $$\implies t = v(n-t)$$ $$\implies \hat{v}=\frac{t}{n-t}$$ Taking the second derivative of our log-likelihood with respect to $v$ yields $\hat{v}=\frac{t}{n-t}$ as our MLE. Now, Bias is defined for any estimator of a parameter $\tau(\theta)$, say $W(\boldsymbol{X})$, to be $$E[W(\boldsymbol{X})]-\tau(\theta)$$ In our case, we're dealing with odds ratio. My question: how do we find the expectation of $\hat{v}$? I've tried using approximation formulas but it leads me to obtain zero bias.


2 Answers 2


This looks like a bit of a trick question. There is a non-zero chance that $\sum X_i=n$, in which case $\frac{t}{n-t}$ is infinite, so the expectation (and thus bias) is also infinite.

  • $\begingroup$ I apologize for taking so long to give this a check mark. It's been busy few weeks. You are absolutely right, sir. We look at the actual expectation, plug in n, and boom we diverge to infinity. I was working waaaaay too hard at this point and very much overthought this question. $\endgroup$ Commented Oct 7, 2015 at 2:54

You can use a taylor expansion of $v=v(p)$ at $p_{0}$, i.e.,$$v(p)=v(p_{0})+\dot{v}(p_{0})(p-p_{0})+\frac{\dot{v}(p_{0})}{2}(p-p_{0})^{2}+\mathcal{R}_{3}$$ When we take $p$ at the MLE $\hat{p}=\frac{\sum x_{i}}{n}$, $\hat{v}=v(\hat{p})$ (by invariability of MLE), we have $$\mathbf{E}(\hat{v})-v(p_{0})=\frac{1}{(1-p_{0})^{3}}\mathbf{Var}(\hat{p})$$ Obviously an unbiased estimator of $v$ does not exist


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.