# Are discrete mixtures Gauss quadrature-like integral approximations?

I noticed that the formula for Gauss (or Newton-Cotes) quadrature looks very similar to the formula for the PDF of a general mixture distribution.

Let $$p_{comp}(x)$$ be the PDF of a compound distribution given by the integral:

$$p_{comp}(x)=\int_{\Theta} p_{param}(\theta)p(x, \theta)d\theta$$

Here we're essentially integrating $$p(x, \theta)$$ w.r.t. the parameter $$\theta$$ using the weighting function $$p_{param}(\theta)$$ which happens to be the PDF of the random parameter $$\theta$$.

Apply numerical integration like Gauss quadrature:

$$p_{comp}(x) \approx \sum_{k=1}^K w_k p(x,\theta_k)$$

This already looks exactly like the PDF of a mixture, but we need to restrict the weights $$w_k$$ first:

1. Since numerical integration formulas must integrate polynomials up to some degree exactly, the first condition is that the "mixture" formula must integrate $$\theta^0=1$$ with weight $$p_{param}(\theta)$$ exactly:

$$\int_{\Theta} p_{param}(\theta)\times 1 d\theta = 1 = \sum_{k=1}^K w_k \times 1$$

The integral equals to one because the weight function $$p_{param}(\theta)$$ is a PDF. Thus, the weights must sum to one.

2. It's known that all weights $$w_k$$ of a Gaussian quadrature formula must be positive: $$w_k > 0 \quad\forall k$$. Even if we used Newton-Cotes, negative weights seem to be frowned upon since they introduce numerical instabilities.

Hence, the weights $$w_k$$ must be a discrete probability distribution (a probability mass function), mirroring the fact that $$p_{param}(\theta)$$ is also a probability distribution (a probability density function).

Having approximated the original compound distribution using numerical integration, we have:

• $$p_{comp}(x) \approx \sum_{k=1}^K w_k p(x,\theta_k)$$
• $$\sum_{k=1}^K w_k = 1$$
• $$w_k > 0 \quad\forall k$$

...which is the definition of a discrete mixture distribution.

So, in principle, if we knew the mixing distribution $$p_{param}(\theta)$$, we could've used Gauss quadrature to approximate the mixture PDF with considerable precision using the standard formula for the PDF of a mixture.

## Question

Can the formula for discrete mixture distributions be derived in this manner? Does it make sense to say something like this?

We know that the PDF of a compound distribution is the integral $$\int_{\Theta} p_{param}(\theta)p(x, \theta)d\theta$$. It can be approximated using Gauss quadrature: $$\int_{\Theta} p_{param}(\theta)p(x, \theta)d\theta \approx \sum_{k=1}^K w_k p(x,\theta_k)$$

Apply the conditions that the weights must be positive and that the sum must at least integrate the constant $$1$$ exactly, show that the weights $$w_k$$ essentially form a discrete probability distribution.

Since the only thing we know about the mixing PDF $$p_{param}(\theta)$$ is that it's a PDF, we can't find neither the weights $$w_k$$ nor the nodes $$\theta_k$$ using Gaussian quadrature because it requires computation of moments of this unknown latent distribution: $$\int_{\Theta}p_{param}(\theta) \theta^n d\theta$$. Thus, we have to resort to statistical estimation methods like maximum likelihood.

Does this make any sense? Can one think about discrete mixtures this way?