2
$\begingroup$

Suppose that we have a mixture model:

$$ p_\theta(y) = \sum_{k = 1}^{K}w_k \phi(y;\mu_k, \sigma^2_k) $$

where $\phi(y;\mu_k, \sigma^2_k)$ is the normal density at $y$ with mean $\mu$ and variance $\sigma^2$. $\theta$ contains the weights, means, and variances.

In this case and in more general cases aside from the Normal mixture, must the model sum or integrate to $1$?

$\endgroup$
3
  • 7
    $\begingroup$ You define it in terms of a probability distribution. Does a probability distribution need to integrate to 1 ..? $\endgroup$
    – Tim
    Commented May 2, 2017 at 7:05
  • 1
    $\begingroup$ I see, so $\int p_\theta(y) = \int \sum_{k = 1}^{K}w_k \phi(y;\mu_k, \sigma^2_k) = \sum_{k = 1}^{K}\int w_k \phi(y;\mu_k, \sigma^2_k) = \sum_{k = 1}^{K} w_k \int \phi(y;\mu_k, \sigma^2_k) = \sum_{k = 1}^{K} w_k = 1$ $\endgroup$
    – user321627
    Commented May 3, 2017 at 0:38
  • 2
    $\begingroup$ If the weights sum to 1, the combined model integrates to 1. $\endgroup$ Commented May 3, 2017 at 6:35

1 Answer 1

2
$\begingroup$

Recall that by the law of total probability

$$ \Pr(A)=\sum_n \Pr(A\mid B_n)\Pr(B_n) $$

and by the axioms of probability $\sum_k \Pr(A_k) = \Pr(\Omega) = 1$, so

$$ \sum_k \sum_n \Pr(A_k\mid B_n)\Pr(B_n) = 1 $$

In the case of mixture distribution

$$ f(x) = \sum_{i=1}^n \pi_i f_i (x|\theta_i) $$

where $\pi_i\ge 0$ and $ \sum_{i=1}^n \pi_i=1$, it can be described in terms of conditional distribution of two random variables

$$ I \sim \mathcal{Cat}(\pi_1,\dots,\pi_n) \\ X_i \sim f_i(\theta_i) $$

and from here you should see right away how the law of total probability applies. What follows, if $f(x)$ is a proper probability distribution it needs to integrate to unity

$$ \int \, f(x) \, dx = \int \, \sum_{i=1}^n \pi_i f_i (x|\theta_i) \, dx = 1 $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.