4
$\begingroup$

I am currently studying on Bayesian models, and still new to probability theory.

I learned that Gaussian Mixture Model is used to represent the distribution of given population as a weighted sum of Gaussian distribution.

What I am curious about is, if the probability density function $f(x)$ is given, can we approximate the function as the sum of the infinite series of gaussians? In other words, are there are ways of finding infinite series of $\alpha$ $\mu$ $\sigma$ for any given pdf $f(x)$ which satisfies:

$$ f(x) = \sum_{i=0}^\infty \alpha_i g_x(\mu_i,\sigma_i) $$ where $$ g_x(\mu,\sigma) = \frac{1}{\sqrt{2\pi} \sigma}\exp[-\frac{1}{2}(\frac{x-\mu}{\sigma})^2] $$ and $$ \sum_{i=0}^\infty \alpha_i = 1 $$

If cannot, I am also curious if there is any way of approximating probability density function as such way. I was thinking of Taylor approximation or Fourier approximation, but they are not the series of probability density function, which should have nonnegative value for every domain and the integral over all values of the random variable must equal one.

Sorry if my question is not good enough. This is my first time using StackExchange.

Edit: I have found that GMM can be act as an universal approximator for distribution on weak topology, but I am still curious if there is any universal approximator for distribution in stronger manner. In other words, is there any pdf $\phi_\theta(x)$ which for any given pdf $f(x)$, there exists infinite series of $\alpha$ $\theta$ which satisfies: $$ f(x) = \sum_{i=0}^\infty \alpha_i \phi_{\theta_i}(x) $$ where $$ \sum_{i=0}^\infty \alpha_i = 1 $$

Edit2:

I should add the condition that $f(x)>0$ for all $x$

$\endgroup$
8
  • 1
    $\begingroup$ Welcome to Cross Validated. I answered a very similar question here and provide a few references. Hope that helps $\endgroup$
    – deemel
    Jul 14 at 6:04
  • 2
    $\begingroup$ @esh3390 This post states that it converges in distribution, but not e.g. in a "stronger" sense of total variation. $\endgroup$
    – frank
    Jul 15 at 9:09
  • 1
    $\begingroup$ @frank thank you for the comment! $\endgroup$
    – esh3390
    Jul 15 at 11:15
  • 2
    $\begingroup$ There is a simple reason why your stronger condition cannot hold: any gaussian pdf satisfies $g_x(\mu_i,\sigma_i)>0$ for every x, yet it could be the case that $f(x)=0$ for some x. Thus you may want to require that $f(x)>0$ for all x. $\endgroup$
    – Mike Hawk
    Jul 15 at 15:36
  • 1
    $\begingroup$ You can construct just about any density as limit of mixtures of Gaussians provided you allow for negative coefficients. However, as I learned long ago to my chagrin (I tried to use mixtures of Gaussians to approximate electronic wave functions in atoms), Gaussians are so smooth and flat near their peaks that it is essentially hopeless to attempt to approximate any density (or any function at all) that has a cusp or discontinuity. You need too many terms. $\endgroup$
    – whuber
    Jul 16 at 17:07

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy