3
$\begingroup$

I am a bit confused about the terminology used in the context of sampling of populations. The Horvitz-Thompson estimator, as well as the Hansen-Hurwitz estimator, for example, are examples of estimators introduced to deal with various sampling methods.

However, I don't understand why they are considered to be estimators in the strictest sense of the word. An estimator is a function that maps the sample space to a set of sample estimates, so to speak. This means that if the following function (the Horvitz-Thompson estimator) is an estimator

$\hat{Y}_{HT}=\sum_{i=1}^{n}\pi_{i}^{-1}Y_{i}$

Then $(Y_{1},...,Y_{n})$ is a random sample and therefore $Y_{i}$ is apparently a random variable. Nevertheless, in this context, the $Y_{i}$ are not treated as random variables (as they are, for example, in books on statistical inference). Precisely because they are not random variables, when proving the unbiasedness of the Horvitz-Thompson estimator, one has to introduce some kind of supporting random variable. Note (https://en.wikipedia.org/wiki/Horvitz%E2%80%93Thompson_estimator), for instance, the difference between the definition of the Horvitz-Thompson estimator when introduced formally, and the definition of the Horvitz-Thompson estimator when proving the unbiasedness of the estimator for the mean.

How is this tension solved?

$\endgroup$
3
  • $\begingroup$ Could you please explain how the $Y_i$ "are not treated as random variables"? And what do you mean by a "supporting" random variable? I suspect you might be trying to ask about how one models samples from finite populations using indicator random variables, but I'm not sure about that. $\endgroup$
    – whuber
    Jun 26, 2023 at 21:49
  • $\begingroup$ When I click on your Wikipedia link, I see: "Formally, let $Y_{i},i=1,2,\ldots ,n$ be an independent sample from..." , which looks like it is being treated as a random variable, to me at least. $\endgroup$
    – jbowman
    Jun 27, 2023 at 0:25
  • $\begingroup$ Let's see if I can be clearer. When trying to prove, for example, the unbiasedness of the HT estimator one uses the indicator function in such a way that $\sum_{i=1}^{n}\pi_{i}^{-1}Y_{i}=\sum_{i=1}^{N}\pi_{i}^{-1}Y_{i}1_{\in D_{n}}$. My doubt arises in the following part of the proof: $\sum_{i=1}^{N}\mathbb{E}[\pi_{i}^{-1}Y_{i}1_{\in D_{n}}]$. My point is that if the $Y_{i}$ are RV, one cannot then infer (at least in general) that $\sum_{i=1}^{N}\pi_{i}^{-1}Y_{i}\mathbb{E}[1_{\in D_{n}}]$. One, of course, perform this step if the $Y_{i}$ are constants, but this is precisely my question. $\endgroup$
    – pompeu
    Jun 27, 2023 at 5:56

2 Answers 2

6
$\begingroup$

You need to handle the population explicitly to get clearer notation, then we can work backwards.

Suppose we have a fixed population of size $N$, with $Y_1,\dots,Y_N$ being the (non-random) values of $Y$. The randomness comes when we select a sample of size $n$. Let $R_i$ be the indicator than observation $i$ was sampled, and write $\pi_i=E[R_i]$ for the probability that $i$ was sampled.

The population total is $T_Y=\sum_{i=1}^N Y_i$ and the Horvitz-Thompson estimator is $$\hat T_Y= \sum_{i=1}^N \frac{R_i}{\pi_i}Y_i.$$

Since the only randomness is in $R$, $$E[\hat T_Y] =E\left[\sum_{i=1}^N \frac{R_i}{\pi_i}Y_i\right]=\sum_{i=1}^N E\left[\frac{R_i}{\pi_i}\right]Y_i.$$

Now $E[R_i/\pi_i]=1$ by the definition of $\pi_i$ and you are done.

The problem with your original notation was that it hid the randomness. If you write $$\hat T= \sum_{i=1}^n \pi_i^{-1}Y_i$$ there now is randomness in both $\pi_i$ and $Y_i$, because $\pi_1$ now means the sampling probability for the first observation we sampled and $Y_1$ now means the value of the first observation we sampled.

When I don't want to explicitly introduce $N$ and $R$ I will usually write $$\hat T= \sum_{i\in\text{sample}} \pi_i^{-1}Y_i$$ which at least indicates that the $i$ depend on the sample. Or you could write $$\hat T= \sum_{i=1}^n \pi_{I_i}^{-1}Y_{I_i}$$ where $I_i$ means the population index of the $i$th value that was sampled.

But it's usually easier just to be explicit about the population if you want to do any inferential reasoning. Save the use of indices over the sample for computational formulas (if anything)

$\endgroup$
3
  • $\begingroup$ (1/2) I think I follow you. So if I understand correctly, indexation can represent either an ordering of the values in the population or an ordering of the observed values. In the first case, the $Y_{i}$ are not random quantities, while in the other case, the $Y_{i}$ are random variables. $\endgroup$
    – pompeu
    Jun 27, 2023 at 8:31
  • $\begingroup$ (2/2) In some books, I have seen an equality stated when proving the unbiasedness of the HT estimator: $\mathbb{E}[\sum_{i=1}^{n}\pi_{i}^{-1}Y_{i}]=\mathbb{E}[\sum_{i=1}^{N}\pi_{i}^{-1}Y_{i}R_{i}]$. I imagine that this is an abuse of notation: the $Y_{i}$ on the left side of the equality are random quantities (and their indexation represents the ordering of the observed values), while the $Y_{i}$ on the right side of the equality are not random quantities (and their indexation represents some kind of ordering of the values in the population). $\endgroup$
    – pompeu
    Jun 27, 2023 at 8:31
  • $\begingroup$ Yes, that sort of abuse of notation is surprisingly difficult to avoid when you are talking about both computation and inference, and it's not confusing to the writer (who knows what they mean) but it can easily be confusing to the reader $\endgroup$ Jun 27, 2023 at 21:14
5
$\begingroup$

Those are, in all sense, legit estimators.

What is probably confusing you is the employment of what you termed "some kind of supporting random variable".

The most conspicuous thing to be noted is you are working with a finite population: you are aware of the sampling frame. The "supporting random variables" that you are referring to are nothing but inclusion indicators - $\mathbb I_i(s) $ indicates whether the index $i$ is present in the sample $s.$ In fact the random vector $\mathbf d:=\langle \mathbb I_i\rangle_{i=1}^N; N:=\operatorname{card}(U),$ plays a pivotal role in probability sampling as $\rm[I]$ writes:

The probabilistic behavior of functions of the sample depends on the probability distribution of $\bf d$.

Any design linear estimator can be expressed as $\hat{\theta}=\sum_{i\in S} w_iy_i, ~w_i$ being independent with respect to the random mechanism generating $y_i$s. The $Y_i$s in a sample is random. But you can take advantage of the finiteness of the population to assess the moments in that $\hat{\theta}$ is basically $\sum_{i\in U} w_iy_i\mathbb I_i(s). $ Of course, when you operate $\mathbb E[\cdot], $ it would be on $\mathbb I_i(s) $ (which would yield the inclusion probability $\pi_i$) that is you are averaging over all possible samples based on the finite population.

Just because you are utilizing the advantage of having a finite population using the inclusion variables doesn't make $\hat{\theta}$ any less of an estimator - it is still a (measurable) function of the sample $s. $


Reference:

$\rm [I]$ Sampling Statistics, Wayne A. Fuller, John Wiley & Sons, $2009,$ sec. $1.2,$ pp. $3-9.$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.