In sampling literature and causal inference literature, there usually is a distinction made about how to view observed data. The first is usually to view some observed data as having come from a finite sample, that is, the same is the entire population. The second is to view the observed data as a random sample from some hypothetical infinite-sized population.

I am wondering what deeper implications are behind thinking about a random sample from a infinitely sized population. Is the infinite part refer to the fact we have convergence results at our disposal? Why not assume a very very large, but finite result? Such that if $N$ is our sample size, then we require the population size $M$ to be $N << M < \infty$.

I feel there is a deeper intuition to assuming a random sample from an infinite population that fundamentally differs from the finite population. What might that be?

  • 1
    $\begingroup$ It's hard to understand the distinction you make initially because no literature, AFAIK, conceives of a sample as being the entire population. Moreover, there is no need to invoke infinities: the process of sampling with replacement from a finite population (as small as two elements) is a sufficiently rich model for most sampling situations. $\endgroup$
    – whuber
    Commented Apr 23, 2019 at 21:42
  • 1
    $\begingroup$ There are convergence theorems for very large populations, such as this handy one showing the hypergeometric distribution converges to the binomial distribution as N goes to infinity. A glance at that proof might suggest a secondary reason why sampling without replacement is not the the default: the resulting distributions are much uglier and harder to work with. $\endgroup$
    – olooney
    Commented Apr 23, 2019 at 21:57
  • $\begingroup$ Often in polling and similar situations, one hopes that the same individual will not be interviewed twice. It is often considered OK to disregard the difference between sampling with and without replacement if sample size $n$ is suitably smaller than population size $N.$ Some authors say $n < .1N.$ Others use $.05,$ a few use $.2$ instead of $.1.$ In any case, the idea is that hypergeometric probabilities have 'nearly' converged to their binomial limits. // Assuming independence from one draw to the next, as in the binomial model can be very convenient, even if slightly inaccurate. $\endgroup$
    – BruceET
    Commented Apr 24, 2019 at 2:51
  • $\begingroup$ In causal inference, is the sampling assumed to be with replacement? Is it fair to summarize the set-up in such a way as to make a direct appeal to Glivenko-Cantelli (the inherent iid sampling procedure)? $\endgroup$
    – user321627
    Commented Apr 24, 2019 at 4:51

1 Answer 1


Comment (continued):

Here are PDFs (PMFs) for samples of size $n=10$ assuming $\mathsf{Binom}(n = 10, p=.3)$ [sampling with replacement or 'infinite' population], compared with hypergeometric distributions with $N = 100$ and $1000$ also with success probability $0.3.$ [Ignore row numbers in brackets.] Means of all three distributions are $\mu =3.$ The binomial variance is $\sigma^2 = 10(.3)(.7) = 2.1.$ The other two variances are shown below the table. (Can you find the formula for the variance of a hypergeometric distribution in your text or notes? Perhaps see Wikipedia on 'finite population correction'.)

k = 0:10;  pdf.bin = dbinom(k, 10, .3)
pdf.h100 = dhyper(k, 30, 70, 10)
pdf.h1000 = dhyper(k, 300, 700, 10)
round(cbind(k, pdf.h100, pdf.h1000, pdf.bin), 4)  # rounding to 4 places
       k pdf.h100 pdf.h1000 pdf.bin
 [1,]  0   0.0229    0.0277  0.0282
 [2,]  1   0.1127    0.1203  0.1211
 [3,]  2   0.2372    0.2339  0.2335
 [4,]  3   0.2812    0.2682  0.2668
 [5,]  4   0.2076    0.2008  0.2001
 [6,]  5   0.0996    0.1026  0.1029
 [7,]  6   0.0315    0.0363  0.0368
 [8,]  7   0.0064    0.0087  0.0090
 [9,]  8   0.0008    0.0014  0.0014
[10,]  9   0.0001    0.0001  0.0001
[11,] 10   0.0000    0.0000  0.0000

sum(pdf.bin*(k - 3)^2)
[1] 2.1                      # binomial
sum(pdf.h100*(k - 3)^2)
[1] 1.909091                 # hypergeometric N = 100
sum(pdf.h1000*(k - 3)^2)
[1] 2.081081                 # hypergeometric N = 1000

Variances for hypergeometric distributions are smaller than for the 'corresponding' binomial distribution. Intuitively, this is because there are less-variable choices when sampling without replacement as the population becomes smaller.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.