Let I have three random variables whose density is a mixture of two Normals with these parameters:

  1. $\mu_{1,1}=6.8$, $\mu_{1,2}=6.95$, $\sigma_{1,1}=0.065$, $\sigma_{1,2}=0.055$ and $\alpha_{1}=0.4$
  2. $\mu_{2,1}=5.7$, $\mu_{2,2}=5.92$, $\sigma_{2,1}=0.08$, $\sigma_{2,2}=0.09$ and $\alpha_{2}=0.3$
  3. $\mu_{3,1}=4.9$, $\mu_{3,2}=5.01$, $\sigma_{3,1}=0.04$, $\sigma_{3,2}=0.1$ and $\alpha_{3}=0.2$

$\alpha_{i}$ is the weight of the first density for variable $i$.

Moreover, I know that those random variables have this correlation matrix (I know it's positive semi-definite, you can change it for numerical examples purposes):

$$\textbf{P}=\begin{bmatrix} 1 & 0.3 & -0.4 \\ 0.3 & 1 & -0.1 \\ -0.4 & -0.1 & 1 \end{bmatrix}$$

I would like to generate correlated random numbers from those mixtures. If you could provide R code, this would be a strong plus.

  • $\begingroup$ I don't understand the distribution you're assuming. Aren't the above parameters suggestion you have a mixture of 3 distributions for two random variables? It looks like your full pdf would be given by: $\alpha_{1}N(x_{1};\mu_{1,1},\sigma_{1,1}) N(x_{2};\mu_{1,2},\sigma_{1,2})+\alpha_{2}N(x_{1};\mu_{2,1},\sigma_{2,1}) N(x_{2};\mu_{2,2},\sigma_{2,2})+\alpha_{3}N(x_{1};\mu_{3,1},\sigma_{3,1}) N(x_{2};\mu_{3,2},\sigma_{3,2})$ ? If it's a mixture of two normals, what's the third $\alpha$ for ? $\endgroup$ – gazza89 Sep 27 '18 at 10:49
  • $\begingroup$ Sorry for the ambiguity, the first distribution is made up by two Normal densities: $\alpha_{1}N(\mu_{1,1},\sigma_{1,1})+(1-\alpha_{1})N(\mu_{1,2},\sigma_{1,2})$. The second random variable follows a distribution made up by two Normal densities: $\alpha_{2}N(\mu_{2,1},\sigma_{2,1})+(1-\alpha_{2})N(\mu_{2,2},\sigma_{2,2})$. And so forth. Actually my notation was not the simplest. $\endgroup$ – Lisa Ann Sep 27 '18 at 11:04
  • $\begingroup$ In that case, why aren't all three variables mutually independent ? Where is the correlation coming from? $\endgroup$ – gazza89 Sep 27 '18 at 11:07
  • $\begingroup$ Uhm... Let I have estimated those mixture parameters by an EM algorithm and by using separated samples. Then I have taken those separated samples and estimated their correlation somehow (e.g. rank). Isn't this theoretically consistent? $\endgroup$ – Lisa Ann Sep 27 '18 at 11:16
  • $\begingroup$ If you assume the model you've assumed, you've baked in feature independence. Your EM algorithm will maximise the likelihood of your data given a model, but if you can see that your features are correlated, you should probably assume a different model and run EM. E.g. $\alpha _{1}N(\underline{x}; \underline{\mu}_{1}, \underline{\underline{\Sigma}}_{1} ) + (1-\alpha _{1})N(\underline{x}; \underline{\mu}_{2}, \underline{\underline{\Sigma}}_{2} ) $, i.e. a superposition of multivariate Gaussians $\endgroup$ – gazza89 Sep 27 '18 at 12:52

This answer applies to the case when the mixture weights are the same for all three coordinates. [I have no idea about a solution in the general case.]

You need to identify a three dimensional Normal mixture which global covariance matrix is Q. This means finding $\mathbf{Q}_1$ and $\mathbf{Q}_2$ such that $$\alpha(\mathbf{Q}_1+\mu_{\cdot 1}\mu_{\cdot 1}^\text{T})+ (1-\alpha)(\mathbf{Q}_2+\mu_{\cdot 1}\mu_{\cdot 2}^\text{T})= \mathbf{Q}+(\alpha\mu_{\cdot 1}+(1-\alpha)\mu_{\cdot 2})(\alpha\mu_{\cdot 1}+(1-\alpha)\mu_{\cdot 2})^\text{T})$$ The number of unknowns in this equation are 3 correlations in $\mathbf{Q}_1$ and 3 correlations in $\mathbf{Q}_2$, since the diagonal terms are given by the marginal Normals. Hence there is a range of choices, provided $\mathbf{P}$ is an achievable correlation matrix for a mixture.

Generating from a multivariate Normal mixture is straightforward: select the component with probability $\alpha$ versus $1-\alpha$ and generate the associated multivariate Normal.

  • $\begingroup$ Could you please expand a bit? Some questions: (1) what would there be on the main diagonal of $\mathbf{Q}$? In the case of multivariate Normal there would be the variance; in the case of Normal mixture, it's hard to say what would there be on the diagonal. (2) Does a Gaussian Copula approach work like you said above? So I can apply the inverse Normal to the quantiles of the Normal mixtures and get a multivariate Normal with correlation $\mathbf{P}$. $\endgroup$ – Lisa Ann Sep 28 '18 at 9:39
  • 1
    $\begingroup$ (1) the diagonal terms of $\mathbf{Q}_1$, $\mathbf{Q}_2$, $\mathbf{Q}$ are all variances, which are given for the mixture components and a consequence of the mixture representation for the resulting $\mathbf{Q}$ . $\endgroup$ – Xi'an Sep 28 '18 at 11:08
  • 1
    $\begingroup$ (2) This is not a copula solution. The issues with copulas are that (a) this correlation matrix may well be out of reach, (b) even if possible, finding the correlation matrix for the Gaussian copula is a complex numerical issue, because correlation does not remain invariant by inverse cdf transforms. $\endgroup$ – Xi'an Sep 28 '18 at 11:11
  • $\begingroup$ As per the copula approach, what if I could have the copula correlation? Would this make the copula approach effective? $\endgroup$ – Lisa Ann Sep 28 '18 at 14:18
  • 1
    $\begingroup$ The copula approach is correct, provided you can set the correlations right on the intended variates, rather than their Uniform transforms. $\endgroup$ – Xi'an Sep 28 '18 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.