# Does a mixture model need to sum or integrate to $1$?

Suppose that we have a mixture model:

$$p_\theta(y) = \sum_{k = 1}^{K}w_k \phi(y;\mu_k, \sigma^2_k)$$

where $\phi(y;\mu_k, \sigma^2_k)$ is the normal density at $y$ with mean $\mu$ and variance $\sigma^2$. $\theta$ contains the weights, means, and variances.

In this case and in more general cases aside from the Normal mixture, must the model sum or integrate to $1$?

• You define it in terms of a probability distribution. Does a probability distribution need to integrate to 1 ..?
– Tim
Commented May 2, 2017 at 7:05
• I see, so $\int p_\theta(y) = \int \sum_{k = 1}^{K}w_k \phi(y;\mu_k, \sigma^2_k) = \sum_{k = 1}^{K}\int w_k \phi(y;\mu_k, \sigma^2_k) = \sum_{k = 1}^{K} w_k \int \phi(y;\mu_k, \sigma^2_k) = \sum_{k = 1}^{K} w_k = 1$ Commented May 3, 2017 at 0:38
• If the weights sum to 1, the combined model integrates to 1. Commented May 3, 2017 at 6:35

Recall that by the law of total probability

$$\Pr(A)=\sum_n \Pr(A\mid B_n)\Pr(B_n)$$

and by the axioms of probability $\sum_k \Pr(A_k) = \Pr(\Omega) = 1$, so

$$\sum_k \sum_n \Pr(A_k\mid B_n)\Pr(B_n) = 1$$

In the case of mixture distribution

$$f(x) = \sum_{i=1}^n \pi_i f_i (x|\theta_i)$$

where $\pi_i\ge 0$ and $\sum_{i=1}^n \pi_i=1$, it can be described in terms of conditional distribution of two random variables

$$I \sim \mathcal{Cat}(\pi_1,\dots,\pi_n) \\ X_i \sim f_i(\theta_i)$$

and from here you should see right away how the law of total probability applies. What follows, if $f(x)$ is a proper probability distribution it needs to integrate to unity

$$\int \, f(x) \, dx = \int \, \sum_{i=1}^n \pi_i f_i (x|\theta_i) \, dx = 1$$