Multinomial distribution - sampling with replacement In multinomial distribution there is a condition that sampling should be done with replacement. What is the real logic behind this?
 A: It is because we want the probabilities of each outcome to be fixed.
The multinomial distribution can be thought of as rolling a weighted $k$-sided die $n$ times where each surface of the die has a fixed probability of coming up. For each roll of the die the probability of any given surface coming up is the same. No matter how many rolls we have made there is no change. Each roll is also independent because the probabilities for the next roll are unaffected by the results of the previous rolls. 
This is like sampling with replacement (SWR): no matter how many samples we have taken, the inclusion probabilities remain unchanged. Compare this with sampling without replacement where the inclusion probability for every remaining element of the population is affected by each draw. 
To make the sampling analogy more clear: suppose that we have $k=3$ categories with probabilities $p_1 = 1/2$, $p_2 = 1/3$, and $p_3 = 1/6$. We can simulate draws from this multinomial by putting 12 tickets in a box where 6 are red (for group 1), 4 are blue (for group 2), and 2 are green (for group 3). If we randomly draw 10 tickets with replacement we have simulated 10 draws from this multinomial distribution because each draw is independent and $P(\textrm{draw red}) = p_1$ for every draw and etc. for the other colors. But if we draw even one ticket without replacement then we no longer have that $P(\textrm{draw red}) = p_1$ for the future draws (because of a lack of independence) so this will not simulate the desired multinomial.
