0
$\begingroup$

I just would like to understand some information about the joint normality and the margins. I read that the normal joint distribution almost always implies that the univariate margins are all normal.

My question is:

1- Is this apply for Gaussian mixture models? For example, in the Gaussian mixture model, each mixture component follows a Gaussian distribution. Hence, does it imply that each margin will be a Gaussian distribution as well? "I understand that mixture components are used to describe the dependency structures, and they are the joint distribution. is that correct?

2- I read this from one answer to a question in this site here. "So yes, the assumption of joint normality is a sufficient condition for all marginal distributions to be normal, irrespective of the dependence structure. Hence, the theory of Copulas does not affect this result in any way."

However, I really do not understand this: irrespective of the dependence structure. As I understand if the joint distribution in normal then the dependence structure is normal.

Could someone help me with my questions, please?

$\endgroup$
8
$\begingroup$

Multivariate Gaussian mixtures are not themselves multivariate Gaussian, their components are.

Statements that apply to the components of a mixture don't generally apply to the mixture (this would be a fallacy of composition, a bit like saying "you have mixed red and blue, which are both primary colours so what you get must be a primary colour").

Multivariate Gaussian mixtures don't have normal margins except in a few special/degenerate cases (like a single component, or where you have an infinite number of components where the means are drawn from a normal distribution and the covariance matrix is constant across components).

As an example, consider a 50-50 mixture of two unit-variance uncorrelated-bivariate-normal components with mean at (-2,-2) and (2,2) respectively. Both margins are bimodal (indeed they have the same distribution in this example); clearly they aren't Gaussian.

plot of large sample from (X,Y) and histogram of marginal

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you so much for your clear answer. Is there a reasons that the normal assumption of the joint implies the that all marginals are normal does not apply to the mixture models? Also, could you please help me with the second question? $\endgroup$ – Maryam Jul 31 '18 at 5:12
  • $\begingroup$ The first question in your comment is completely addressed by the first 8 words of my answer. The part of your post under your point "2." doesn't actually contain any questions (there's nothing to answer there), but I believe your lack of understanding there is again completely covered by the first 8 words of my answer. Why would you expect properties of multivariate normals to hold for things that are not multivariate normal? $\endgroup$ – Glen_b Jul 31 '18 at 11:25
  • $\begingroup$ Thank you so much for your nice comment. My supervisors told me that Gaussian models can model the dependences structures of non-linear and also can be deal with non-gaussian margins and the assumption of normality does not make sense. $\endgroup$ – Maryam Jul 31 '18 at 12:27
  • $\begingroup$ If you mean Gaussian mixture models (which are not themselves Gaussian models), then yes, they can model nonlinear dependence and they do have non-Gaussian margins (as you see in the simple example in the answer). $\endgroup$ – Glen_b Jul 31 '18 at 12:54
  • $\begingroup$ But this source, researchgate.net/publication/…, said GMM only assumed gaussian margins. (please see the introduction). $\endgroup$ – Maryam Jul 31 '18 at 12:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.