Sampling from a multivariate Gaussian mixture model How can I generate multi-dimensional data from a (estimated) Gaussian mixture pdf? In general, what would be ways to generate multi-dimensional data from a pdf? I read rejection sampling can be used, but it requires estimating an upper bound for the pdf, which is not an easy task for me. 
 A: A Gaussian mixture model with $k$ mixture components can be expressed as:
$$p(x) = \sum_{i=1}^k \pi_i f(x \mid \mu_i, C_i)$$
where $\pi_i, \mu_i, C_i$ are the mixture weight, mean, and covariance matrix of the $i$th mixture component and $f_i$ is a multivariate normal distribution.
To sample a point from the GMM, first choose a mixture component by drawing $j$ from the categorical distribution with probabilities $[\pi_1, \dots, \pi_d]$. This can be done using a random number generator for the categorical distribution. Alternatively, draw a random number $r$ from the uniform distribution on $[0, 1]$, then find the smallest integer $j$ such that $r \le \sum_{i=1}^j \pi_i$.
Then sample from the $j$th mixture component using a multivariate Gaussian random number generator with mean $\mu_j$ and covariance matrix $C_j$. This can also be done manually. Let $\Lambda_j$ be a diagonal matrix containing the eigenvalues of $C_j$ and $V_j$ be a matrix containing the eigenvectors of $C_j$ along its columns. Generate a column vector $y$ with entries drawn i.i.d. from the standard normal distribution. Then:
$$x = V_j \Lambda_j^{\frac{1}{2}} y + \mu_j$$
Sampling from a GMM is convenient because it can be done using widely available random number generators. Sampling from multivariate distributions in general is not so straightforward. In the best case, one can find tricks to make it easier for specific distributions. In the worst case, one must resort to techniques like MCMC.
