I was reading the Wikipedia example from the Multinomial Distribution article:

In it they say:

"Note: Since we’re assuming that the voting population is large, it is reasonable and permissible to think of the probabilities as unchanging once a voter is selected for the sample. Technically speaking this is sampling without replacement, so the correct distribution is the multivariate hypergeometric distribution, but the distributions converge as the population grows large."

I am confused because doesn't sampling without replacement, indicate that the probabilities will change once a voter is selected from the sample?

  • 4
    $\begingroup$ Yes, it does. But if the original population is really large, they change by such a tiny amount that "it is reasonable and permissible to think of the probabilities as unchanging once a voter is selected for the sample". The multinomial (sampling with replacement) and multivariate hypergeometric (sampling without replacement) distributions converge as the population grows larger, so there's a miniscule-to-no benefit to using the more complex multivariate hypergeometric with a large population. $\endgroup$ – jbowman Mar 8 '18 at 20:33
  • $\begingroup$ Hi. Thanks. I just got confused when it is written that the probabilities are unchanging, and then they write, "Technically speaking, this is sampling without replacement"... It's probably then, just how they have worded it. $\endgroup$ – Thomas Moore Mar 8 '18 at 20:36
  • 3
    $\begingroup$ Yes, it's not the clearest explanation I've ever seen, and can be read two ways. $\endgroup$ – jbowman Mar 8 '18 at 20:37

As said in comments, this is sampling without replacement, which we can model with the (multivariate) hypergeometric distribution, see https://en.wikipedia.org/wiki/Hypergeometric_distribution#Multivariate_hypergeometric_distribution. Then if the total population size (and each of the subpopulation sizes) becomes very large compared to the sample size, this can be effectively approximated by the multinomial distribution, which models sampling with replacement.

This can be shown formally using a limit argument, where we nees Stirling's approximation for the factorial: $$ n! \sim \sqrt{2\pi n} \left( \frac{n}{e} \right)^n $$ where the tilde ($\sim$) means asymptotically equal, that is, the quotient of the two sides converge to 1 when $n\to \infty$. Let the total population size be $N$ and the subpopulation sizes be $N_i$, we have $\sum_i N_i =N$. The sample size is $n$ and the sample numbers from the subpopulations is $n_i$, $\sum_i n_i=n$.

Now we assume that the population sizes goes to infinity $N \to \infty, N_i \to\infty$ in such a way that $\frac{N_i}{N} \to p_i$, $\sum_i p_i=1$. Taking limits, the sample size $n$ is a constant.

We can write the (multivariate) hypergeometric mass function as $$ \frac{\prod_i \binom{N_i}{n_i}}{\binom{N}{n}} $$ First, write out the binomial coefficients as factorials, rearranging and using Stirling's formula, noting that the $\sqrt{2\pi}$ and $(\frac1e)^\cdot$ things will cancel immediately, we get $$ \frac{\prod_i \frac{N_i!}{n_i! (N_i-n_i)!} }{\frac{N!}{n! (N-n)!} } = \binom{n}{n_1 n_2 \dotso}\times \prod_i \frac{N_i^{1/2}N_i^{N_i} }{N^{1/2}}N^N \times \prod_i \frac{(N-n)^{1/2} (N-n)^{N-n}}{(N_i-n_i)^{1/2}(N_i-n_i)^{N_i-n_i}} $$ Now,taking limits, $\frac{N_i}{N_i-n_i} \to 1$ and $\frac{N}{N-n} \to 1$. So the squareroot terms will disappear in the limit. What we have left then is $$ \binom{n}{n_1 n_2 \dotso} \times \prod_i\left( \frac{N_i}{N} \right)^{N_i} \times \prod_i \left( \frac{N-n}{N_i-n_i} \right)^{N_i-n_i} $$ Taking limits, rearranging and cancelling we get the mass function for the multinomial distribution $$ \binom{n}{n_1 n_2 \dotso} \prod_i p_i^{n_i} $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.