4
$\begingroup$

Let I have three random variables whose density is a mixture of two Normals with these parameters:

  1. $\mu_{1,1}=6.8$, $\mu_{1,2}=6.95$, $\sigma_{1,1}=0.065$, $\sigma_{1,2}=0.055$ and $\alpha_{1}=0.4$
  2. $\mu_{2,1}=5.7$, $\mu_{2,2}=5.92$, $\sigma_{2,1}=0.08$, $\sigma_{2,2}=0.09$ and $\alpha_{2}=0.3$
  3. $\mu_{3,1}=4.9$, $\mu_{3,2}=5.01$, $\sigma_{3,1}=0.04$, $\sigma_{3,2}=0.1$ and $\alpha_{3}=0.2$

$\alpha_{i}$ is the weight of the first density for variable $i$.

Moreover, I know that those random variables have this correlation matrix (I know it's positive semi-definite, you can change it for numerical examples purposes):

$$\textbf{P}=\begin{bmatrix} 1 & 0.3 & -0.4 \\ 0.3 & 1 & -0.1 \\ -0.4 & -0.1 & 1 \end{bmatrix}$$

I would like to generate correlated random numbers from those mixtures. If you could provide R code, this would be a strong plus.

$\endgroup$
5
  • $\begingroup$ I don't understand the distribution you're assuming. Aren't the above parameters suggestion you have a mixture of 3 distributions for two random variables? It looks like your full pdf would be given by: $\alpha_{1}N(x_{1};\mu_{1,1},\sigma_{1,1}) N(x_{2};\mu_{1,2},\sigma_{1,2})+\alpha_{2}N(x_{1};\mu_{2,1},\sigma_{2,1}) N(x_{2};\mu_{2,2},\sigma_{2,2})+\alpha_{3}N(x_{1};\mu_{3,1},\sigma_{3,1}) N(x_{2};\mu_{3,2},\sigma_{3,2})$ ? If it's a mixture of two normals, what's the third $\alpha$ for ? $\endgroup$
    – gazza89
    Sep 27, 2018 at 10:49
  • $\begingroup$ Sorry for the ambiguity, the first distribution is made up by two Normal densities: $\alpha_{1}N(\mu_{1,1},\sigma_{1,1})+(1-\alpha_{1})N(\mu_{1,2},\sigma_{1,2})$. The second random variable follows a distribution made up by two Normal densities: $\alpha_{2}N(\mu_{2,1},\sigma_{2,1})+(1-\alpha_{2})N(\mu_{2,2},\sigma_{2,2})$. And so forth. Actually my notation was not the simplest. $\endgroup$
    – Lisa Ann
    Sep 27, 2018 at 11:04
  • $\begingroup$ In that case, why aren't all three variables mutually independent ? Where is the correlation coming from? $\endgroup$
    – gazza89
    Sep 27, 2018 at 11:07
  • $\begingroup$ Uhm... Let I have estimated those mixture parameters by an EM algorithm and by using separated samples. Then I have taken those separated samples and estimated their correlation somehow (e.g. rank). Isn't this theoretically consistent? $\endgroup$
    – Lisa Ann
    Sep 27, 2018 at 11:16
  • $\begingroup$ If you assume the model you've assumed, you've baked in feature independence. Your EM algorithm will maximise the likelihood of your data given a model, but if you can see that your features are correlated, you should probably assume a different model and run EM. E.g. $\alpha _{1}N(\underline{x}; \underline{\mu}_{1}, \underline{\underline{\Sigma}}_{1} ) + (1-\alpha _{1})N(\underline{x}; \underline{\mu}_{2}, \underline{\underline{\Sigma}}_{2} ) $, i.e. a superposition of multivariate Gaussians $\endgroup$
    – gazza89
    Sep 27, 2018 at 12:52

1 Answer 1

1
$\begingroup$

This answer applies to the case when the mixture weights are the same for all three coordinates. [I have no idea about a solution in the general case.]

You need to identify a three dimensional Normal mixture which global covariance matrix is Q. This means finding $\mathbf{Q}_1$ and $\mathbf{Q}_2$ such that $$\alpha(\mathbf{Q}_1+\mu_{\cdot 1}\mu_{\cdot 1}^\text{T})+ (1-\alpha)(\mathbf{Q}_2+\mu_{\cdot 1}\mu_{\cdot 2}^\text{T})= \mathbf{Q}+(\alpha\mu_{\cdot 1}+(1-\alpha)\mu_{\cdot 2})(\alpha\mu_{\cdot 1}+(1-\alpha)\mu_{\cdot 2})^\text{T})$$ The number of unknowns in this equation are 3 correlations in $\mathbf{Q}_1$ and 3 correlations in $\mathbf{Q}_2$, since the diagonal terms are given by the marginal Normals. Hence there is a range of choices, provided $\mathbf{P}$ is an achievable correlation matrix for a mixture.

Generating from a multivariate Normal mixture is straightforward: select the component with probability $\alpha$ versus $1-\alpha$ and generate the associated multivariate Normal.

$\endgroup$
5
  • $\begingroup$ Could you please expand a bit? Some questions: (1) what would there be on the main diagonal of $\mathbf{Q}$? In the case of multivariate Normal there would be the variance; in the case of Normal mixture, it's hard to say what would there be on the diagonal. (2) Does a Gaussian Copula approach work like you said above? So I can apply the inverse Normal to the quantiles of the Normal mixtures and get a multivariate Normal with correlation $\mathbf{P}$. $\endgroup$
    – Lisa Ann
    Sep 28, 2018 at 9:39
  • 1
    $\begingroup$ (1) the diagonal terms of $\mathbf{Q}_1$, $\mathbf{Q}_2$, $\mathbf{Q}$ are all variances, which are given for the mixture components and a consequence of the mixture representation for the resulting $\mathbf{Q}$ . $\endgroup$
    – Xi'an
    Sep 28, 2018 at 11:08
  • 1
    $\begingroup$ (2) This is not a copula solution. The issues with copulas are that (a) this correlation matrix may well be out of reach, (b) even if possible, finding the correlation matrix for the Gaussian copula is a complex numerical issue, because correlation does not remain invariant by inverse cdf transforms. $\endgroup$
    – Xi'an
    Sep 28, 2018 at 11:11
  • $\begingroup$ As per the copula approach, what if I could have the copula correlation? Would this make the copula approach effective? $\endgroup$
    – Lisa Ann
    Sep 28, 2018 at 14:18
  • 1
    $\begingroup$ The copula approach is correct, provided you can set the correlations right on the intended variates, rather than their Uniform transforms. $\endgroup$
    – Xi'an
    Sep 28, 2018 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.