# How to find an unbiased estimator?

Suppose $X_1, X_2, ...,X_n$ are samples from a uniform discrete distribution with probability 1/3 on each of the points $\theta-1, \theta, \theta+1$, where $\theta\in\mathbb{Z}.$ From "Theory of point Estimation" by Lehman and Casella. Page130 Problem 1.9

I want to find all unbiased estimators of zero. How should I do that?

My understanding:
The definition of unbiased estimator says, $E[\delta(x)]=g(\theta)$.

which means, I want anything that has $bias=E[\delta(x)]-g(\theta)=0$

But how exactly do I find this guy, $\delta(x)$? And what is the parameter of interest here?

My work:
$f(x;3)= \frac{1}{3}, \text{for } \theta-1\le x \le \theta+1, \text{where }\theta\in\mathbb{Z}$

Could someone point out what I am missing here? I feel that I am pretty close to the answer.

• Presumably, you are you looking for an unbiased estimator of $\theta$ and not an estimator of $0$ (as stated). The latter is a known constant with the trivially unbiased estimator $\delta(x)=0$. If you wish to find an unbiased estimator $\theta$, a good place to start is to compute $E(X_i)$. Oct 8, 2012 at 5:28
• @MånsT: I am look for "all unbiased estimators of zero". I don't understand that question. And also, what is $\theta$ here? I edited the question. Oct 8, 2012 at 5:59
• Hint: if you have 3 distinct sample values, you know that $\theta$ is the middle value. Oct 8, 2012 at 6:08
• I know this is obvious to most, but several comments seem to have created some confusion: characterizing the unbiased estimators of zero is of interest because once you have obtained a single unbiased estimator of $g(\theta)$, you get all the rest by adding an unbiased estimator of zero to it. So, we are free to view this question from either point of view: characterizing all unbiased estimators of $g(\theta)$ or all unbiased estimators of zero.
– whuber
Oct 8, 2012 at 15:48

If the assignment asks for all estimators, then probably the correct answer would be to list a property which the desired estimator must satisfy. The estimator is simply the function $\delta:\mathbb{R}^n\to \mathbb{R}$. The requirement is that it should be unbiased:

$$E_\theta\delta(X_1,...,X_n)=0.$$

Now vector $(X_1,...,X_n)$ assumes value $(x_{i_1},...,x_{i_n})$ with probability $(1/3)^n$, where $(x_{i_1},...,x_{i_n})\subset \{\theta-1,\theta,\theta+1\}^n$. So you can write down how the expectation looks like and you'll get a new condition of what $\delta$ must satisfy. Maybe it will give you some additional ideas how to proceed.

• +1. The key is that this new condition must hold no matter what value $\theta$ has. It can also help to find the answer explicitly for the case $n=1$.
– whuber
Oct 8, 2012 at 15:51
• I am confused about why do you have $(1/3)^n$. Actually, why use likelihood? I thought unbiasedness was all about expectation. Likelihood was for MLE. Oct 8, 2012 at 22:24

The answer is simple, any statistic $$\delta(X)$$ satisfying $$\delta(\theta-1)+\delta(\theta)+\delta(\theta+1)=0, \forall\theta\in\mathbb{Z}$$ is an unbiased estimator of zero. Thus, the totality of the unbiased estimators of zero is given by

$$\left\{\delta\in\{\text{measurable functions from }\mathbb{Z}\text{ to }\mathbb{R}\},\delta(\theta-1)+\delta(\theta)+\delta(\theta+1)=0, \forall\theta\in\mathbb{Z}\right\}$$

Actually you need to solve the following equation: $$Ef(X_1,\dots,X_n)=\theta.$$ For example if $f(x_1,\dots,x_2)=\sum_{i=1}^n \lambda_i x_i$ then $$E\sum_{i=1}^n \lambda_i X_i=\sum_{i=1}^n \lambda_i EX_i=\theta.$$ Then $EX_1=\dots=EX_n=1/3\cdot((\theta-1)+(\theta)+(\theta+1))=\theta$, and mind that $EX_i$ only equals to $\theta$ if all three probabilities are the same! Then we insert $EX_i=\theta$ and get $$\sum_{i=1}^n \lambda_i \theta= \theta \sum_{i=1}^n \lambda_i=\theta.$$ Now we see that if $\sum_{i=1}^n \lambda_i=1$ we have unbiased estimator of $\theta$, for example $X_1$ is unnbiased estimator yielding $\lambda_1=1$ and $\lambda_i=0, \forall i \neq 1$.

So these are all linear unbiased estimators and what about non linear? Like median? Is there some additional info that we could be missing in you assignment?

• The question really does ask for "all unbiased estimators of zero". I don't know if it's a mistake. Oct 8, 2012 at 6:42
• so probably you have to set $\theta=0$ Oct 8, 2012 at 6:44
• @jem77bfp: how did you get that? Oct 8, 2012 at 22:58
• You said "all unbiased estimators of zero" so I assumed that real $\theta$ must be equal to zero. Oct 9, 2012 at 6:18