Parameterizing finite mixture distribution

Question

Let's consider a finite mixture:

$$f(x) = \sum_{i=1}^{N}w_{i}p_{i}\left(x\right)$$

where:

• $N$ is the number of mixed distributions
• $\left\{p_{1},\dots, p_{N}\right\}$ is a finite set of one-dimensional probability density functions in $\mathbb{R}^{+}$
• $\left\{w_{1},\dots, w_{N}\right\}$ is a finite set of associated weights

Now, let's say that $f$ is a mixture over a parametric family, where distributions are parameterized by a set of $\left\{\alpha^{1},\dots, \alpha^{n}\right\}$ parameters (where the upper index is just an index and not an exponentiation):

$$f(x) = \sum_{i=1}^{N}w_{i}p\left(x;\alpha^{1}_{i},\dots, \alpha^{n}_{i}\right)$$

And let's say that these parameters can be themselves parameterized by a single parameter:

$$f(x, \lambda) = \sum_{i=1}^{N}w_{i}p_{i}\left(x;\alpha^{1}_{i}\left(\lambda\right),\dots, \alpha^{n}_{i}\left(\lambda\right)\right)$$

---

The question is: is there a distribution $p$ such that $f$ itself would be a member of the parametric family:

$$p\left(x;\alpha^{1}_{0},\dots, \alpha^{n}_{0}\right) = \sum_{i=1}^{N}w_{i}p\left(x;\alpha^{1}_{i}\left(\lambda\right),\dots, \alpha^{n}_{i}\left(\lambda\right)\right)$$

In other words, given:

• a mixture size $N$
• a parameter $\lambda$
• a set of parameters $\left\{\alpha^{1}_{0},\dots, \alpha^{n}_{0}\right\}$

would there be a distribution $p$ such that it would be possible to find a set of weights $w_{i} \neq 0$ and a set of set of parameters $\left\{\left\{\alpha^{1}_{1}\left(\lambda\right),\dots, \alpha^{n}_{1}\left(\lambda\right)\right\}, \dots, \left\{\alpha^{1}_{N}\left(\lambda\right),\dots, \alpha^{n}_{N}\left(\lambda\right)\right\}\right\}$ so that the equality above holds.

---

For example if $p$ is a normal distribution, can one find a set of $N$ normal distributions and their weights parameterized by $\lambda$ so that the mixture is itself a normal distribution?

Answer 1

If we ignore some trivial solutions such that $w_1=1$ and $w_i=0$ for the rest or $a_i=a_j$ for all $i,j$, then the only solution seems to be histograms with the same binning. With weights, you simply sum up the weights in particular bins. They guarentee the linearity.

Histograms (not necessarily 1D only) can be used for approximation of any distribution with arbitrary precision and they do not have to be equidistant. You can even consider some weird histograms defined on top of e.g. Voronoi mosaic or any other splitting of the input space.