# Finding a minimum variance unbiased (linear) estimator

Here is a basic question that perhaps has a simple answer, but one that I was not able to find by quickly scanning the literature.

Suppose that I have a collection of $n$ unopened boxes. Each box $i$ contains either a red ball or a blue ball: write $c_i = 1$ if box $i$ contains a red ball, and $c_i = 0$ otherwise. Moreover, each box $i$ has a probability $p_i > 0$ written on it. If I attempt to open box $i$, then with probability $p_i$ I will succeed and observe whether $c_i = 1$ or $c_i = 0$. With probability $1-p_i$ I will fail, and will not learn $c_i$. I can only try once -- if the box does not open, I never learn $c_i$. The parameters for each box $(c_i, p_i)$ are drawn i.i.d from an uknown distribution $\mathcal{D}$. I wish to produce an unbiased estimate of the expected fraction of red balls:$$C \equiv E_{(c_i, p_i) \sim \mathcal{D}}[ c_i]$$ Note that while the marginal distribution on the $c_i$ is Bernoulli, I make no assumptions about how the $p_i$ are distributed -- they could be correlated with the $c_i$ in some arbitrary way.

An obvious first attempt at finding an unbiased estimator is to attempt to open each box, and set $d_i = \frac{c_i}{p_i}$ if the box opens, and $d_i = 0$ otherwise. Then take as my estimate: $$S \equiv \frac{1}{n}\sum_{i=1}^nd_i$$ Since $E[d_i] = C$ for each $i$, this is an unbiased estimator. Can I argue that it is the minimum variance unbiased estimator?

If I was literally sampling the values $d_i$, then I could at least say that $S$ was the minimum variance linear unbiased estimator, since each $d_i$ is sampled i.i.d and I am estimating the sample mean. But can I say anything about $S$ given that I am actually sampling $(c_i, p_i)$? Is there a literature on this sort of problem?

-
add comment

## 1 Answer

Your setup is analogous to sampling from a finite population (the $c_i$) without replacement, with a fixed probability $p_i$ of selecting each member of the population for the sample. Successfully opening the $i^{th}$ box corresponds to selecting the corresponding $c_i$ for inclusion in the sample.

The estimator you describe is a Horvitz-Thompson estimator, which is the only unbiased estimator in the class of estimators $\hat{S} = \sum_{i=1}^{N} \beta_i c_i$, where $\beta_i$ is a weight to be used whenever $c_i$ is selected for the sample. Thus, within that class of estimators, it is also the optimal unbiased estimator regardless of the criterion for optimality.

It has been shown (Ramakrishnan) that the H-T estimator is admissible in the class of all unbiased estimators of a finite population total. (Note the link is not to the original paper by Godambe and Joshi, which I can't seem to find online.)

For a review of the Horvitz-Thompson estimator and its properties, see Rao.

-
Thank you -- this is just what I was looking for! –  Aaron Dec 8 '11 at 22:38
add comment