How are estimators like the Horvitz-Thompson Estimator derived?

Question

The Horvitz-Thompson Estimator is usually given by:

$$ \hat{Y}_{HT} = \sum_{i=1}^n \pi_i ^{-1} Y_i $$

The proof that it is unbiased is trivial to do. In additional, there exists other estimators out there for different designs as well, like those in Rubin and Rosenbaum (1983). However, in each of the original papers, the estimator seemed to appear out of nowhere with no motivation, only appearing so that the author could show it was unbiased.

My question is, is there a solid way to come up with unbiased estimators like there?

Answer 1

the estimator seemed to appear out of nowhere with no motivation

If you think the idea behind stratified sampling is intuitive, then I believe Horvitz-Thompson should come as a natural extension, it's not something out of the blue.

To illustrate how a simple stratified sample could help you come up with the formula, consider a case with two strata, $S_1$ and $S_2$ of known sizes $N_1$ and $N_2$ and suppose you get samples $n_1$ and $n_2$ respectively. Now imagine you compute the average for each sample $\bar{y}_1$ and $\bar{y}_2$.

How would you use this information to estimate the total $Y$? The natural way is to take the estimated average of each stratum and multiply by the total (population) number of elements of the stratum:

$$ \hat{Y} = N_1 \bar{y}_1 + N_2 \bar{y}_2 $$

But take this simple expression and rewrite it as:

$$ \begin{align} \hat{Y} &= N_1 \sum_{i \in S_1}\frac{y_i}{n_1} + N_2 \sum_{i \in S_2}\frac{y_i}{n_2}\\ &= \sum_{i \in S_1}\frac{y_i}{n_1/N_1} + \sum_{i \in S_2}\frac{y_i}{n_2/N_2}\\ &= \sum_{i} \frac{y_i}{\pi_i} \end{align} $$

Where $\pi_i = n_1/N_1$ if $i\in S_1$ and $\pi_i = n_2/N_2$ if $i\in S_2$. That is, a simple stratified sampling already gives you the insight that, in essence, what we are doing is summing each sampled $y_i$ upweighted by its probability of selection. Then the idea of a general inverse probability weighting, where each $\pi_i$ could be different, should come naturally.

Answer 2

I'd like to take a different approach to the accepted answer. The accepted answer justifies the intuition behind the inverse probability weighting, but I'd like to justify why the Horvitz-Thompson Estimator makes sense to arrive at rather than some other estimator. It can stem from two properties:

1. The estimator is a linear combination of survey responses. This property is an advantage because it allows us to calculate expectations (and variances) without considering all possible samples, and instead only deal with individual units' (joint) probabilities of selection.

2. The estimator is unbiased (as you already note). This is a nice property because it makes the eventual estimate easier to interpret/explain to someone else.

The Horvitz-Thompson Estimator is the unique estimator with these two properties. In my opinion, the most intuitive way to approach it is to:

1. Justify the two properties above,

2. Use property 1 to define the structure of the estimator ($\sum w_i y_i$),

3. Use property 2 to show that $w_i=\pi_i^{-1}$, and then,

4. Show that this is unique (e.g. consider particular $y_i$ values)

Answer 3

First, note that the Horvitz-Thompson (H-T) estimator is used for both $\textit{with or without replacement}$ random sampling designs.

$\pi$ is the inclusion probability. The H-T estimator provides us with an estimate of the population total.

As a simple illustration, a Horvitz-Thompson "like" estimator can be derived as follows:

If we let $\pi = \frac{n}{N}$, where $n$ is the sample size and $N$ is the population size, and substitute into the H-T estimator, then, after simplification, we get that $\hat{Y} = N\frac{Y}{n} = N\bar{y}$.

Showing that such an estimator is unbiased is now a straightforward task.