How can I generate multi-dimensional data from a (estimated) Gaussian mixture pdf? In general, what would be ways to generate multi-dimensional data from a pdf? I read rejection sampling can be used, but it requires estimating an upper bound for the pdf, which is not an easy task for me.

  • 1
    $\begingroup$ @Tim I don't think this is a duplicate because that question was about univariate distributions, whereas this one is about multivariate $\endgroup$
    – user20160
    Mar 22 '17 at 21:50

A Gaussian mixture model with $k$ mixture components can be expressed as:

$$p(x) = \sum_{i=1}^k \pi_i f(x \mid \mu_i, C_i)$$

where $\pi_i, \mu_i, C_i$ are the mixture weight, mean, and covariance matrix of the $i$th mixture component and $f_i$ is a multivariate normal distribution.

To sample a point from the GMM, first choose a mixture component by drawing $j$ from the categorical distribution with probabilities $[\pi_1, \dots, \pi_d]$. This can be done using a random number generator for the categorical distribution. Alternatively, draw a random number $r$ from the uniform distribution on $[0, 1]$, then find the smallest integer $j$ such that $r \le \sum_{i=1}^j \pi_i$.

Then sample from the $j$th mixture component using a multivariate Gaussian random number generator with mean $\mu_j$ and covariance matrix $C_j$. This can also be done manually. Let $\Lambda_j$ be a diagonal matrix containing the eigenvalues of $C_j$ and $V_j$ be a matrix containing the eigenvectors of $C_j$ along its columns. Generate a column vector $y$ with entries drawn i.i.d. from the standard normal distribution. Then:

$$x = V_j \Lambda_j^{\frac{1}{2}} y + \mu_j$$

Sampling from a GMM is convenient because it can be done using widely available random number generators. Sampling from multivariate distributions in general is not so straightforward. In the best case, one can find tricks to make it easier for specific distributions. In the worst case, one must resort to techniques like MCMC.

  • $\begingroup$ Thanks! I was wondering if R has any package doing this? I am thinking about writing a small package to apply your method, but just to make sure I am not reinventing the wheel. $\endgroup$
    – bankrip
    Mar 22 '17 at 21:40
  • $\begingroup$ Glad it helps. Not an R user so can't make any recommendations there $\endgroup$
    – user20160
    Mar 22 '17 at 21:55

Not the answer you're looking for? Browse other questions tagged or ask your own question.