Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am learning Maximum likelihood estimators for a inference class. And this is a problem I came across.

Let $X_1,X_2,X_3,\ldots, X_n$ be a random sample with p.m.f $$p(X)=\theta(1-\theta)^x; x=0,1,2,\ldots\quad \mathrm{and}\quad 0<\theta<1$$.
As maximum Likelihood estimator I obtained $\hat{\theta}=\frac{1}{\bar{x}+1}$.

Now the question asks to obtain a unbiased estimator for $\theta$ when $n$ is large and considering it's distribution for large sample approximate 95% confidence interval for $\theta$.

What I am unable to do is to find a unbiased estimator. Then if $\hat\theta $ is unbiased for $\theta$ using the property that $\hat{\theta}\sim \mathrm{N}\left(\theta,{1\over I_x(\theta)}\right)$. I can construct a confidence interval.

But how can I find a unbiased estimator for $\theta$?
I thought since MLE's are not in general unbiased but are asymptotically unbiased to compute $\mathrm{E}\left(\frac{1}{\bar{x}+1}\right)$.

Here $\sum X_i$ follows a negative binomial distribution right?

Then $\mathrm{E}\left(\frac{1}{\bar{x}+1}\right) =\mathrm{E}\left(\frac{n}{T+n}\right)$ where T=$\sum X_i$.

Then I come up with $\sum n\left(\frac{1}{t+n}\right)\binom{n+t-1}{t} \theta^n(1-\theta)^t $. I am unable to show that this is asymptotically unbiased.

Please help me to find a unbiased estimator for $\theta$ when $n$ is large.

share|improve this question

One unbiased estimator of $\theta$ is $I[X_1=0]$. The joint pmf is $\theta^n(1-\theta)^t$, where $t=\sum_i x_i$ is a sufficient statistic for $\theta$. By the Rao-Blackwell theorem, a better estimator, also unbiased, is $$\begin{aligned} \mathbb{E}\Big[ I[X_1=0] \Big| \sum_i X_i = t \Big] &= \sum_{k_{2:n}\geq0}I\big[{\textstyle\sum} k_i=t\big] \Big/ \sum_{k_{1:n}\geq0}I\big[{\textstyle\sum} k_i=t\big] \\&= \binom{t+n-2}{n-2} \Big/ \binom{t+n-1}{n-1} \\&= \frac{n-1}{t+n-1}, \qquad (n\geq2). \end{aligned}$$

Note also that $T=\sum_i X_i$ is a complete sufficient statistic, so by the Lehmann-Scheffe theorem, $\frac{n-1}{T+n-1}$ is the uniformly minimum variance unbiased estimator for $\theta$. When $n=1$, this is just $I[T=0]$.

share|improve this answer
How is the joint pmf of $I[X_1=0]$ is found.Also using this how would I construct a confidence interval – clarkson Jul 8 '14 at 3:01
@clarkson You can find the confidence interval by setting $Z=\frac1{1+T/(n-1)}$ and calculating $$\mathbb{P}[\theta\in[a Z,b Z]].$$ For very large $n$ you can approximate the distribution of $T$ with a Gaussian. – Kirill Jul 8 '14 at 7:24

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.