5
$\begingroup$

Given the pdf $f(x) = \sum_i \omega_i \mathcal{N}(x; \mu_i, C_i )$ of a gaussian mixture density, where the $i$-th component has mean $\mu_i$ and covariance matrix $C_i$ and the weights $\omega_i$ sum to 1, is there a formula to compute the moments and central moments of this density from the given $\omega_i$'s, $\mu_i$'s and $C_i$'s?

$\endgroup$
7
$\begingroup$

It is simple because of linearity of integration (exchange order of integration and expectation). $\mu=\omega_1 \mu_1 +\omega_2\mu_2 +\dots+\omega_k \mu_k$ The same is true for higher order moments and central moments with the $\mu_i$s replaced by the variances for variances etc. Now since each $\mu_i$ and $C_i$ determines the higher order moments by normality the third and fourth moments for example can all be expressed as functions of them.

Anything else you want to know about finite mixtures can be found in these books(I include the EM Algorithm book because that is the method most often used to get the MLEs for the parameters:

Finite Mixture Models

The EM Algorithm and Extensions

Nonparametric Statistics and Mixture Models: A Festschrift in Honor of Thomas P Hettmansperger

Medical Applications of Finite Mixture Models

Mixture Models

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I am a bit unclear on the variance of $f(x)$. Intuitively I would have thought that for a mixture model, the variance would also depend on the means, and not just be $C=w_1C1+w_2C_2+\dots+w_kC_k$ as your answer implies. There is a related answer on the math SE which is somewhat contradicting. $\endgroup$ – Jakob Feb 15 '13 at 20:39
0
$\begingroup$

I illustrate the calculation with 2 components. The other cases are similar. The key results to use are (a) E(E(X|Y))=E(X) and (b) V(X)= V(E(X|Y))+E(V(X|Y)).

Here Y denotes the component. So Y takes the values 1 and 2 with probabilities p and 1-p.

Let E(X|Y=i) = $\mu_i$ and V(X|Y=i) =$\sigma_i^2$

Now E(X)= p $\mu_1$ + (1-p) $\mu_2$.

V(E(X|Y))= p $(\mu_1 - (p \mu_1 + (1-p) \mu_2)^2$ + (1-p) $(\mu_2 - (p \mu_1 + (1-p) \mu_2)^2$ which on simplification yields p(1-p)$(\mu_1-\mu_2)^2$

E(V(X|Y))= p$\sigma_1^2$ + (1-p)$\sigma_2^2$

Thus, V(X) = p$\sigma_1^2$ + (1-p)$\sigma_2^2$ + p(1-p)$(\mu_1-\mu_2)^2$

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.