# multinomial distribution sampling

I was reading the Wikipedia example from the Multinomial Distribution article:

In it they say:

"Note: Since we’re assuming that the voting population is large, it is reasonable and permissible to think of the probabilities as unchanging once a voter is selected for the sample. Technically speaking this is sampling without replacement, so the correct distribution is the multivariate hypergeometric distribution, but the distributions converge as the population grows large."

I am confused because doesn't sampling without replacement, indicate that the probabilities will change once a voter is selected from the sample?

• Yes, it does. But if the original population is really large, they change by such a tiny amount that "it is reasonable and permissible to think of the probabilities as unchanging once a voter is selected for the sample". The multinomial (sampling with replacement) and multivariate hypergeometric (sampling without replacement) distributions converge as the population grows larger, so there's a miniscule-to-no benefit to using the more complex multivariate hypergeometric with a large population. – jbowman Mar 8 '18 at 20:33
• Hi. Thanks. I just got confused when it is written that the probabilities are unchanging, and then they write, "Technically speaking, this is sampling without replacement"... It's probably then, just how they have worded it. – Thomas Moore Mar 8 '18 at 20:36
• Yes, it's not the clearest explanation I've ever seen, and can be read two ways. – jbowman Mar 8 '18 at 20:37

This can be shown formally using a limit argument, where we nees Stirling's approximation for the factorial: $$n! \sim \sqrt{2\pi n} \left( \frac{n}{e} \right)^n$$ where the tilde ($\sim$) means asymptotically equal, that is, the quotient of the two sides converge to 1 when $n\to \infty$. Let the total population size be $N$ and the subpopulation sizes be $N_i$, we have $\sum_i N_i =N$. The sample size is $n$ and the sample numbers from the subpopulations is $n_i$, $\sum_i n_i=n$.
Now we assume that the population sizes goes to infinity $N \to \infty, N_i \to\infty$ in such a way that $\frac{N_i}{N} \to p_i$, $\sum_i p_i=1$. Taking limits, the sample size $n$ is a constant.
We can write the (multivariate) hypergeometric mass function as $$\frac{\prod_i \binom{N_i}{n_i}}{\binom{N}{n}}$$ First, write out the binomial coefficients as factorials, rearranging and using Stirling's formula, noting that the $\sqrt{2\pi}$ and $(\frac1e)^\cdot$ things will cancel immediately, we get $$\frac{\prod_i \frac{N_i!}{n_i! (N_i-n_i)!} }{\frac{N!}{n! (N-n)!} } = \binom{n}{n_1 n_2 \dotso}\times \prod_i \frac{N_i^{1/2}N_i^{N_i} }{N^{1/2}}N^N \times \prod_i \frac{(N-n)^{1/2} (N-n)^{N-n}}{(N_i-n_i)^{1/2}(N_i-n_i)^{N_i-n_i}}$$ Now,taking limits, $\frac{N_i}{N_i-n_i} \to 1$ and $\frac{N}{N-n} \to 1$. So the squareroot terms will disappear in the limit. What we have left then is $$\binom{n}{n_1 n_2 \dotso} \times \prod_i\left( \frac{N_i}{N} \right)^{N_i} \times \prod_i \left( \frac{N-n}{N_i-n_i} \right)^{N_i-n_i}$$ Taking limits, rearranging and cancelling we get the mass function for the multinomial distribution $$\binom{n}{n_1 n_2 \dotso} \prod_i p_i^{n_i}$$