# Practical considerations on a mixture of Multivariate Normals, with many terms

Let's say the density of $$Y$$ is given by $$p(y)=\frac{1}{L}\sum^L_{i=1}N(y\mid \mu_i, \Sigma_i)$$, where $$N(y \mid \mu_i, \Sigma_i)$$ is the multivariate normal density evaluated at $$y$$, with known $$L,\mu_i,\Sigma_i$$.

1. If I wanted to simulate from $$p(y),$$ could I just take a multinomial with $$L$$ probabilities, and then simulate from the respective $$i$$-th category?
2. If we have a very big $$L,$$ something like $$20,000.$$ What's then a good way to sample from that mixture? When generating a sample from a mixture $$\sum w_i f_i$$, with a given size (N), why not simply take $$N_i$$ draws from each density $$f_i$$, such that $$\frac{N_i}{N}=w_i$$? Can we do it?
3. Is there a closed formula representation for a mixture of multivariate normals?

1. Yes, you can take a multinomial with $$n=1$$ and using a probability vector $$p=\mathbb{1}_{1\times L}\frac{1}{L}$$. Or use Categorical Distribution, which is actually a special case. Another simple method would be generating a uniform random variable $$U$$, in $$[0,1)$$, and choose the category $$\lfloor LU \rfloor$$ (if numbered from $$0$$). In non-equal mixture probabilities this is handled internally in random number generation via slight modification of the binary search idea.
2. It shouldn't matter much when you have $$L=20000$$. But, you can also use the uniform RV approach to get rid of probability vector allocation as well. For your updated extension of Q2, as I've noted in the comments, when split sizes (elements coming from each mixture component) are assumed to be constant, you're not truly sampling the mixture distribution you wrote; even if it can emulate the mixture in large samples.
3. To the best of my knowledge, there is none. I think, trying to write it in a closer form is like trying to write $$e^a+e^b$$ in closer form, in which I don't see any further simplification.
• Gunes, thank you for your answer. I still have a question though. When generating a sample from a mixture $\sum w_i f_i$, with a given size (N), why not simply take $N_i$ draws from each density $f_i$, such that $\frac{N_i}{N}=w_i$? Can we do it? – An old man in the sea. Feb 12 at 17:56
• Oh, bad choice of word. It resembles the biased/unbiased estimation. However, yes, in the long run, your samples will better represent the mixture PDF, but note that you're not truly sampling it. Let's say you've $N=100$ and $2$ mixtures. True samples from this mixture would contain the sample split $[40,60]$ or $[45,55]$ for example, but in your way, you'll always get $[50,50]$. – gunes Feb 12 at 18:24