I am learning Maximum likelihood estimators for a inference class. And this is a problem I came across.

Let $X_1,X_2,X_3,\ldots, X_n$ be a random sample with p.m.f $$p(X)=\theta(1-\theta)^x; x=0,1,2,\ldots\quad \mathrm{and}\quad 0<\theta<1$$.
As maximum Likelihood estimator I obtained $\hat{\theta}=\frac{1}{\bar{x}+1}$.

Now the question asks to obtain a unbiased estimator for $\theta$ when $n$ is large and considering it's distribution for large sample approximate 95% confidence interval for $\theta$.

What I am unable to do is to find a unbiased estimator. Then if $\hat\theta $ is unbiased for $\theta$ using the property that $\hat{\theta}\sim \mathrm{N}\left(\theta,{1\over I_x(\theta)}\right)$. I can construct a confidence interval.

But how can I find a unbiased estimator for $\theta$?
I thought since MLE's are not in general unbiased but are asymptotically unbiased to compute $\mathrm{E}\left(\frac{1}{\bar{x}+1}\right)$.

Here $\sum X_i$ follows a negative binomial distribution right?

Then $\mathrm{E}\left(\frac{1}{\bar{x}+1}\right) =\mathrm{E}\left(\frac{n}{T+n}\right)$ where T=$\sum X_i$.

Then I come up with $\sum n\left(\frac{1}{t+n}\right)\binom{n+t-1}{t} \theta^n(1-\theta)^t $. I am unable to show that this is asymptotically unbiased.

Please help me to find a unbiased estimator for $\theta$ when $n$ is large.


1 Answer 1


One unbiased estimator of $\theta$ is $I[X_1=0]$. The joint pmf is $\theta^n(1-\theta)^t$, where $t=\sum_i x_i$ is a sufficient statistic for $\theta$. By the Rao-Blackwell theorem, a better estimator, also unbiased, is $$\begin{aligned} \mathbb{E}\Big[ I[X_1=0] \Big| \sum_i X_i = t \Big] &= \sum_{k_{2:n}\geq0}I\big[{\textstyle\sum} k_i=t\big] \Big/ \sum_{k_{1:n}\geq0}I\big[{\textstyle\sum} k_i=t\big] \\&= \binom{t+n-2}{n-2} \Big/ \binom{t+n-1}{n-1} \\&= \frac{n-1}{t+n-1}, \qquad (n\geq2). \end{aligned}$$

Note also that $T=\sum_i X_i$ is a complete sufficient statistic, so by the Lehmann-Scheffe theorem, $\frac{n-1}{T+n-1}$ is the uniformly minimum variance unbiased estimator for $\theta$. When $n=1$, this is just $I[T=0]$.

  • $\begingroup$ How is the joint pmf of $I[X_1=0]$ is found.Also using this how would I construct a confidence interval $\endgroup$
    – clarkson
    Jul 8, 2014 at 3:01
  • $\begingroup$ @clarkson You can find the confidence interval by setting $Z=\frac1{1+T/(n-1)}$ and calculating $$\mathbb{P}[\theta\in[a Z,b Z]].$$ For very large $n$ you can approximate the distribution of $T$ with a Gaussian. $\endgroup$
    – Kirill
    Jul 8, 2014 at 7:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.