Derive househould weights from a uniformly distributed person sample

Question

The Swiss Public-Use Sample of the national census is a 5% sample drawn from the entire census survey. According to the documentation, the persons are sampled uniformly without replacement. Persons are the "primary sampling unit", the selection probability and person weights are all equal: $\pi = 0.05, w = 1/\pi = 20$.

For each person there are household-level attributes as well. The sample is not uniformly distributed anymore when considering the households: Larger households have a bigger sampling probability than smaller households. In order to carry ot analyses at the household level (with variables such as "number of children in the household"), the records must be reweighted.

Intuition suggests that each person record $p$ should be weighted with $w_p / k_p$ (with $k_p$ being the number of persons in the household). This should hold exactly if the persons were drawn with replacement, and at least approximately in the "without replacement" scheme. (The sample size is by five orders of magnitude larger than the maximum household size.) However, I find it difficult to provide a formal argument for this.

Is my intuition correct? How would a proof or a probabilistic argument look like? Can you suggest a textbook that treats similar examples, perhaps even this?

Answer 1

I think you should conceptualise your new sample as a weighted sample with replacement of households from the original population of households. Then the weighting scheme comes naturally according to standard principles - weights are proportional to the inverse of the probability of a household being chosen in the sample.

Answer 2

Sampling without replacement is full of various stumbling blocks, often not very intuitive. A concatenation of two samples drawn WITH replacement is a sample with replacement again; a concatenation of two WOR samples is not a WOR sample, for instance.

If you have an SRSWOR sample of size $n$ from the population of size $N$ (American survey statisticians hate the European ones who can just use the population registers!), then the probability of selecting a group of size $k$ (e.g., a household of this size) is

$$ {\rm Prob}[\mbox{at least 1 member of the group is in the sample}] = $$ $$ 1 - {\rm Prob}[\mbox{no member of the group is in the sample}] = $$ $$ 1 - {k \choose 0}{N-k \choose n}\biggl/{N \choose n} = 1 - \frac{(N-k)!(N-n)!}{(N-k-n)!N!} $$ $$ \sim 1 - \frac{(N-k)^{N-k+1/2} (N-n)^{N-n+1/2}}{(N-k-n)^{N-k-n+1/2}N^{N+1/2}} $$ $$ = 1 - \frac{(1-k/N)^{N-k+1/2} (1-n/N)^{N-n+1/2}}{(1-(k+n)/N)^{N-k-n+1/2}} $$ $$ \sim 1 - \frac{\exp(-k)(1-k/N)^{-k+1/2} \exp(-n)(1-n/N)^{-n+1/2}}{\exp(-k-n)(1-(k+n)/N)^{-k-n+1/2}} $$ $$ \sim 1 - \frac{(1+k(k-1/2)/N)(1+n(n-1/2)/N)}{1+(k+n)(k+n-1/2)/N} $$ $$ \sim 1 - \biggl[1+\frac{k(k-1/2)}N +\frac{n(n-1/2)}N - \frac{(k+n)(k+n-1/2)}N\biggr] \sim 2kn/N $$ where the first approximation uses Stirling's approximation for the factorial, the second one, the exponential limit (I cheated with the power 1/2 though), and the subsequent ones, the Taylor series expansion of $(1+x)^a$ near $x=0$ that ignores the terms $o(1/N)$. I don't know where the factor of 2 is creeping in, it seems totally out of whack there. The first order approximation to the probability of selection should instead be $kn/N=kf$ where where $f=0.05$ is the (individual) sampling fraction. I'd appreciate it if somebody could point out where I made a mistake. A good sampling book should give correct derivation.

A useful approximation is Poisson sampling with the rate $kn/N$. Under this approximation, the probability of having zero hits is $\exp(-kn/N)$, and the probability of at least one hit is thus $1-\exp(-kn/N) = 1 - \exp(-kf)$. So instead of $w(k)=1/(kf)$, as you initially suggested, you'd want to have $w(k)=\bigl[1-\exp(-kf)\bigr]^{-1}$; for the households of size 10, the difference between the two numbers will be notable.

Answer 3

Following up Peter's answer, here is my solution to the problem.

A given household of size $k$ must be reweighted using $w = 1/E[K]$ with $K$ being a random variable that gives the number of persons sampled for the household. The variable $K_i$ is approximately distributed like $k$ independent Bernoulli trials with success probability $\pi$ each, its expecation is $E[k] = k\pi$. My initial intuition was correct! This is even more surprising given that the sampling probabilities of a household follow a hypergeometric distribution which can be approximated by a bionomial distribution, and the expected value of this distribution is beautiful again. (Even more: The mean = expected value of the hypergeometric distribution is also $k\pi$ in our case -- ho need to approximate at all! This also means that it doesn't matter if the person sample was taken with or without replacement.)

The result is given in Cochran (1977), Chapter 9A. (Not Chapter 11, the results there are similar but the underlying sampling scheme is different.) There, they call "clusters" what are "households" in our case. However, this book derives the weights directly, without taking expectation values of membership counts. These expected values are mentioned in a 2009 lecture by James R. Chromy, Some Generalizations of the Horvitz-Thompson Estimator. (So, nothing really new...)

---

Following up Stas' answer, here is an alternative approach to deriving the sampling probaility for selecting no member of a group of size $k$:

$$ {\rm Prob}[\mbox{at least 1 member of the group is in the sample}] = \\ 1 - {\rm Prob}[\mbox{no member of the group is in the sample}] = 1 - \pi_0 = \\ 1 - {k \choose 0}{N-k \choose n}\biggl/{N \choose n} = 1 - \frac{(N-k)!(N-n)!}{(N-k-n)!N!} \\ \pi_0 = \frac{(N-k)!}{N!}\frac{(N-n)!}{(N-k-n)!} = \prod_{i=1}^{k}\frac{i+N-k-n}{i+N-k} \\ \approx \left(1 - \frac{n}{N}\right)^k \\ 1 - \pi_0 \approx 1 - \left(1 - \frac{n}{N}\right)^k $$

The approximation uses $k \ll n$ and $\frac{n}{N - k} \approx \frac{n}{N}$.