Similar to how you can derive the normal distribution as the distribution where the probability near a point exponentially decays with the square of the number of standard deviations you are away from the mean I was wondering if its possible to derive a similar distribution with multiple different means $\mu_i$ each with its own standard deviation $\sigma_i$ where probability exponentially decays with the square of the number of standard deviations to the closest mean i.e. the density is

$$f(x)\propto exp(-(x-\mu_k)^2/(2\sigma_k^2))$$

where $\mu_k$ is the closest mean in terms of the number of standard deviations $\sigma_k$ from x to $\mu$.

Is it possible to integrate the expression above and find a normalisation coefficient?

It sounds a lot like a Gaussian mixture model which I've briefly read about.

If the probability exponentially decays with distance to the closest mean in terms of standard deviations then I believe you will get a normally shaped distribution around each mean and when you are equally close to two distributions in terms of standard deviations then the probability is equally likely that you came from either component and this is where the density will switch from one Gaussian shape to a Gaussian shape centred at another point.

Does that make sense? Basically I'm wondering if this characterisation of a density function that exponentially decays with the number of standard deviations to the closest $\mu_k$ is equivalent to the normal definition of a Gaussian mixture model where you have a linear combination of Gaussians where the coefficients are between zero and one.

EDIT: Another idea I had was imagine if instead of a finite number of discrete means you had a parametric curve such that the probability density exponentially decays with the number of standard deviations to the closest point on the curve. Now that definitely doesn't seem like a traditional Gaussian mixture model.


1 Answer 1


When accounting for the proximity constraint, the density looks like $$f(x)\propto\sum_{k=1}^K\exp\{-(x-\mu_k)^2/2\sigma_k^2\}\mathbb I_{(x-\mu_k)^2/\sigma_k^2=\min_{1\le j\le K} (x-\mu_j)^2/\sigma_j^2}$$ where one and only one term in the summation is different from zero.

It is thus a mixture of truncated Normal distributions, with weights $$\dfrac{\sqrt{2\pi}\sigma_k}{K}\,\mathbb P_{\mu_k,\sigma_k}\left\{(X-\mu_k)^2/\sigma_k^2=\min_{1\le j\le K} (X-\mu_j)^2/\sigma_j^2\right\}$$ However, since the supports of the components of the mixture never intersect (once again, there is only one term different from zero in the summation), there is no uncertainty on the component to which allocate an observation (and no free weight) and it is better seen as a piecewise-normal distribution. It is definitely not a Normal mixture and the normalising weights are computable as shown above.

Here is an illustration of the resulting density of the distribution when $K=3$, $\mu_k=-2,0,3$ and $\sigma_k=2,1/2,1$:

enter image description here

It is clear how the density is piecewise Normal around each of the three means $\mu_k$.

  • $\begingroup$ Oh, okay so that's an indicator term, I wasn't sure what it was. It makes a lot more sense now. In terms of the exponential error I must've been hurriedly writing things down and mistakenly have thought exp(a+b)=exp(a)+exp(b). $\endgroup$ Commented Jun 18, 2021 at 12:52
  • $\begingroup$ What does the P notation mean in the expression for the weights? Did you derive the weights by integrating the density? $\endgroup$ Commented Jun 18, 2021 at 13:00
  • $\begingroup$ $\mathbb P$ is rather traditional for denoting the probability (of the event between curly brackets). Here it is computed for a Normal distribution with mean $\mu_k$ and sd $\sigma_k$. As you guessed, this probability is exactly the integral of the exponential term over the event (subset of the real line), so that the [reweighted] truncated Normal density has a mass of one. The $1/K$ is added for the overall integral to be equal to one. $\endgroup$
    – Xi'an
    Commented Jun 18, 2021 at 13:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.