Let's consider a finite mixture:

$$f(x) = \sum_{i=1}^{N}w_{i}p_{i}\left(x\right)$$


  • $N$ is the number of mixed distributions
  • $\left\{p_{1},\dots, p_{N}\right\}$ is a finite set of one-dimensional probability density functions in $\mathbb{R}^{+}$
  • $\left\{w_{1},\dots, w_{N}\right\}$ is a finite set of associated weights

Now, let's say that $f$ is a mixture over a parametric family, where distributions are parameterized by a set of $\left\{\alpha^{1},\dots, \alpha^{n}\right\}$ parameters (where the upper index is just an index and not an exponentiation):

$$f(x) = \sum_{i=1}^{N}w_{i}p\left(x;\alpha^{1}_{i},\dots, \alpha^{n}_{i}\right)$$

And let's say that these parameters can be themselves parameterized by a single parameter:

$$f(x, \lambda) = \sum_{i=1}^{N}w_{i}p_{i}\left(x;\alpha^{1}_{i}\left(\lambda\right),\dots, \alpha^{n}_{i}\left(\lambda\right)\right)$$

The question is: is there a distribution $p$ such that $f$ itself would be a member of the parametric family:

$$p\left(x;\alpha^{1}_{0},\dots, \alpha^{n}_{0}\right) = \sum_{i=1}^{N}w_{i}p\left(x;\alpha^{1}_{i}\left(\lambda\right),\dots, \alpha^{n}_{i}\left(\lambda\right)\right)$$

In other words, given:

  • a mixture size $N$
  • a parameter $\lambda$
  • a set of parameters $\left\{\alpha^{1}_{0},\dots, \alpha^{n}_{0}\right\}$

would there be a distribution $p$ such that it would be possible to find a set of weights $w_{i} \neq 0$ and a set of set of parameters $\left\{\left\{\alpha^{1}_{1}\left(\lambda\right),\dots, \alpha^{n}_{1}\left(\lambda\right)\right\}, \dots, \left\{\alpha^{1}_{N}\left(\lambda\right),\dots, \alpha^{n}_{N}\left(\lambda\right)\right\}\right\}$ so that the equality above holds.

For example if $p$ is a normal distribution, can one find a set of $N$ normal distributions and their weights parameterized by $\lambda$ so that the mixture is itself a normal distribution?

  • $\begingroup$ I would think that the circumstances would be bizarre. For example, if one adds sub-populations of normal distributions with the same means and variances the result is normal. However, there is no way to fit a mixture model to a normal distribution to recreate the sub-populations from which it arose. $\endgroup$ – Carl Oct 19 '18 at 0:11

If we ignore some trivial solutions such that $w_1=1$ and $w_i=0$ for the rest or $a_i=a_j$ for all $i,j$, then the only solution seems to be histograms with the same binning. With weights, you simply sum up the weights in particular bins. They guarentee the linearity.

Histograms (not necessarily 1D only) can be used for approximation of any distribution with arbitrary precision and they do not have to be equidistant. You can even consider some weird histograms defined on top of e.g. Voronoi mosaic or any other splitting of the input space.

  • $\begingroup$ Added $w_{i} \neq 0$ $\endgroup$ – Vincent Oct 18 '18 at 20:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.