# Are linear combinations (i.e. "sums") of gaussian distributions also gaussian?

I have always assumed that this is fact : but are there any mathematical theorems that state: finite sums of gaussian distributions are gaussian themselves?

In the above picture, does p(x) have a gaussian distribution? Do the sums need to be linear? Can they be non-linear?

I saw the following question: Is the joint distribution of two linear combinations of Gaussians still a multivariate normal? - but no reference/proof was provided.

Thanks

• Mixtures of normal distributions need not be normal. Oct 15 '21 at 3:42
• the whole point of mixtures is to model non-Gaussian distributions Oct 15 '21 at 20:17

## 2 Answers

It is the linear combination of jointly Gaussian random variables (RVs) that results in another RV with Gaussian density. In your question, you have linear combination of Gaussian densities; therefore, the resulting density need not be Gaussian.

Below is given a working proof of this theorem. The characteristic function of an RV $$X$$ is $$\phi_X(t)=E(e^{\iota tx}),$$ where $$E$$ denotes expectation. The characteristic function of the linear combination $$Y$$ of two RVs $$X_1$$ and $$X_2$$ is $$Y=\alpha_1X_1+\alpha_2X_2,$$ and the characteristic function of $$Y$$ is $$\phi_Y(t)=E(e^{\iota tY})=E(e^{\iota t\alpha_1X_1 +\iota t\alpha_2X_2}).$$ If $$X_1$$ and $$X_2$$ are independent $$\phi_Y(t)$$ can be written as $$\phi_Y(t) = E(e^{\iota t\alpha_1X_1})E(e^{\iota t\alpha_2X_2}).$$

Now, for a Gaussian RV $$X$$ with mean $$\mu$$ and variance $$\sigma^2$$,$$\phi_X(t) = \exp(\iota t\mu-\frac{1}{2}\sigma^2t^2).$$ It is a bit tideous to derive this result but it can be done with some effort. Following this result, $$\phi_Y(t)=\exp[\iota t(\alpha_1\mu_1+\alpha_2\mu_2)-\frac{1}{2}(\alpha_1^2\sigma_1^2+\alpha_2^2\sigma_2^2)t^2].$$

Now, we also know that characteristic function of a probability distribution is unique (but it is difficult to prove). Following this result, characteristic function of $$Y$$ is clearly that of a Gaussian with mean $$\alpha_1\mu_1 + \alpha_2\mu_2$$ and variance $$\alpha_1^2\sigma_1^2 + \alpha_2^2\sigma_2^2$$. It basically proves that linear combination of two independent Gaussian RVs is a Gaussian RV.

Edit: The proof above is valid only when $$X_1$$ and $$X_2$$ are independent. But a similar proof can be carried out when $$X_1$$ and $$X_2$$ are not independent. Suppose $$X = [X_1,X_2]^T$$ is a jointly Gaussian with mean vector $$\mu = [\mu_1,\mu_2]^T$$ and covariance matrix $$C$$. Also, the characteristic function of a random vector $$X$$ is $$\phi_X(t_1,t_2)=E(e^{\iota t^TX})=E(e^{\iota (t_1X_1+t_2X_2)}),$$

where $$t=[t_1,t_2]^T$$. The characteristic function of an RV $$Y=\alpha_1X_1+\alpha_2X_2$$ is $$\phi_Y(u)=E(e^{\iota uY})=E(e^{\iota u(\alpha_1X_1+\alpha_2X_2)})$$ $$=E(e^{\iota (u\alpha_1X_1+u\alpha_2X_2)}).$$

Notice that, $$\phi_Y(u)=\phi_X(\alpha_1u,\alpha_2u).$$

Now, we know that (the proof is a bit involved): $$\phi_X(t)=\exp[\iota t^T\mu-\frac{1}{2}t^TCt],$$ which implies $$\phi_Y(u)=\exp[\iota u\mu_y-\frac{1}{2}u^2\sigma_y^2],$$ where $$\mu_y=\alpha^T\mu=\alpha_1\mu_1+\alpha_2\mu_2,$$ $$\sigma_y^2=\alpha^TC\alpha$$ $$\alpha=[\alpha_1,\alpha_2]^T.$$

The last four equations show that $$Y$$ is a Gaussian RV with mean $$\mu_y$$ and variance $$\sigma_y^2$$. I have not given the derivation of the last four equations, let me know if that would be helpful.

A side note: The RVs $$X_1$$ and $$X_2$$ should be jointly Gaussian, otherwise this result would not hold. An illustrative example is given by Chris Huang in comments section.

• The first statement is not always true, it depends on their joint distribution. The proof that follows is for a very special case (independence). Oct 15 '21 at 12:08
• @ChrisHaug I will add the proof for a more general case. But what are the conditions on join distribution? Can you please elaborate? Oct 15 '21 at 16:03
• It's true if they are jointly Gaussian. You can have distributions which have Gaussian marginals but are not jointly Gaussian. For example, my answer here: stats.stackexchange.com/questions/238226 . There, $X+Y$ has an atom at zero and so cannot be Gaussian, for example. Oct 15 '21 at 17:21
• @ AbhinavGupta : Thank you so much for your answer! So just to clarify: linear combinations of gaussian densities WILL NOT NECESSAIRLY have a gaussian distribution ... BUT linear combinations of gaussian random variables WILL NECESSAIRLY have a gaussian distribution? Thank you so much! Oct 15 '21 at 17:31
• @stats555 (1) No, the linear combinations of Gaussian densities are not necessarily Gaussian. (2) Linear combinations of JOINTLY Gaussian RVs is necessarily Gaussian. The conditions 'jointly' is important (As Chris Huang has pointed out). I will edit my answer to include this condition. Oct 15 '21 at 20:31

Suppose that you have two random variables $$x\sim F_x$$ and $$y\sim F_y$$ where $$F_x,F_y$$ are probability distributions. Then generally $$w_xx+w_yy$$ is not from $$w_xF_x+w_yF_y$$, even if when $$F_x,F_y$$ are normal distributions.

The characteristic function of the weighted sum is the product of characteristic functions of these distributions: $$\psi_{w_xx+w_yy}(t)=\psi_x(w_xt)\times \psi_y(w_yt)$$. You can obtain the distribution $$F_{w_xx+w_yy}$$ from its characteristic function, of course.

Finally, the whole idea of the Gaussian mixture is to model unknown distributions, such as those with fat tails in finance.

Now, what is meant by $$w_x\mathcal N+w_y\mathcal N$$? In case of a mixture it is the weighted average density (PDF).