Are discrete mixtures Gauss quadrature-like integral approximations?

Question

I noticed that the formula for Gauss (or Newton-Cotes) quadrature looks very similar to the formula for the PDF of a general mixture distribution.

Let $p_{comp}(x)$ be the PDF of a compound distribution given by the integral:

$$ p_{comp}(x)=\int_{\Theta} p_{param}(\theta)p(x, \theta)d\theta $$

Here we're essentially integrating $p(x, \theta)$ w.r.t. the parameter $\theta$ using the weighting function $p_{param}(\theta)$ which happens to be the PDF of the random parameter $\theta$.

Apply numerical integration like Gauss quadrature:

$$ p_{comp}(x) \approx \sum_{k=1}^K w_k p(x,\theta_k) $$

This already looks exactly like the PDF of a mixture, but we need to restrict the weights $w_k$ first:

1. Since numerical integration formulas must integrate polynomials up to some degree exactly, the first condition is that the "mixture" formula must integrate $\theta^0=1$ with weight $p_{param}(\theta)$ exactly:

   $$\int_{\Theta} p_{param}(\theta)\times 1 d\theta = 1 = \sum_{k=1}^K w_k \times 1$$

   The integral equals to one because the weight function $p_{param}(\theta)$ is a PDF. Thus, the weights must sum to one.

2. It's known that all weights $w_k$ of a Gaussian quadrature formula must be positive: $w_k > 0 \quad\forall k$. Even if we used Newton-Cotes, negative weights seem to be frowned upon since they introduce numerical instabilities.

Hence, the weights $w_k$ must be a discrete probability distribution (a probability mass function), mirroring the fact that $p_{param}(\theta)$ is also a probability distribution (a probability density function).

---

Having approximated the original compound distribution using numerical integration, we have:

• $p_{comp}(x) \approx \sum_{k=1}^K w_k p(x,\theta_k)$
• $\sum_{k=1}^K w_k = 1$
• $w_k > 0 \quad\forall k$

...which is the definition of a discrete mixture distribution.

So, in principle, if we knew the mixing distribution $p_{param}(\theta)$, we could've used Gauss quadrature to approximate the mixture PDF with considerable precision using the standard formula for the PDF of a mixture.

Question

Can the formula for discrete mixture distributions be derived in this manner? Does it make sense to say something like this?

We know that the PDF of a compound distribution is the integral $\int_{\Theta} p_{param}(\theta)p(x, \theta)d\theta$. It can be approximated using Gauss quadrature: $$\int_{\Theta} p_{param}(\theta)p(x, \theta)d\theta \approx \sum_{k=1}^K w_k p(x,\theta_k)$$

Apply the conditions that the weights must be positive and that the sum must at least integrate the constant $1$ exactly, show that the weights $w_k$ essentially form a discrete probability distribution.

Since the only thing we know about the mixing PDF $p_{param}(\theta)$ is that it's a PDF, we can't find neither the weights $w_k$ nor the nodes $\theta_k$ using Gaussian quadrature because it requires computation of moments of this unknown latent distribution: $\int_{\Theta}p_{param}(\theta) \theta^n d\theta$. Thus, we have to resort to statistical estimation methods like maximum likelihood.

Does this make any sense? Can one think about discrete mixtures this way?

Answer 1

The problem you have described is related to the theme of "randomization", discussed in On Randomization of Affine Diffusion Processes with Application to Pricing of Options on VIX and S&P 500 or Randomization of Short-Rate Models, Analytic Pricing and Flexibility in Controlling Implied Volatilities. As you can see, the mixture approach gives you quite some flexibility as it allows you to make model parameters random, thus providing additional degrees of freedom. BTW, you can find a GitHub page with the codes for randomizing affine models.