Find the variance-covariance matrix for a linear combination of multiple bivariate normal distribution?

I have an arbitrary number of independnet bivariate normal distributions with $$\mu_i = [x_i,z_i]$$ & $$\Sigma_i= \left(\begin{array}{cc} \sigma^2_{x_i} & \sigma^2_{x_i,z_i}\\\ \sigma^2_{x_i, z_i} & \sigma^2_{z_i} \end{array}\right)$$

Where i is arbitrarily large

I want to take a linear combination of these bivariate normal distributions with weights $$c = [c_1,...,c_i]$$ where $$\sum c_i = 1$$ & $$c_i >0$$

Obviously, the linear combination of $$\mu_{mixture} = [\sum c_ix_i,\sum c_iz_i]$$

However, I am not sure about the linear combination of the variance-covariance matrix.

Does anyone know how I can calculate this pooled & weighted variance-covariance? Looking for the variance-covariance matrix for the mixture distribution.

Thanks so much!

• Your question is ambiguous. Are you interested in properties of (a) the distribution of a linear combination of random variables having these distributions or (b) a mixture of these distributions?
– whuber
Jul 8 '21 at 19:33
• @whuber Interested in the weighted mixture distribution of the i bivariate normal distributions
– CJR
Jul 8 '21 at 19:56
• Please edit your post to state that clearly, because the answer that has been posted starts off with the other interpretation.
– whuber
Jul 8 '21 at 20:33
• @whuber - thanks for catching that. I have edited the post
– CJR
Jul 8 '21 at 20:52
– whuber
Jul 8 '21 at 22:02

Let $$X_i\stackrel{\text{indep}}\sim\mathcal N(\mu_i,\Omega_i)$$ and let $$S = \sum_{i=1}^n c_iX_i$$. A linear combination of independent Gaussians is Gaussian so we just need the mean and covariance. By linearity we have $$\text E[S] = \sum_i c_i\mu_i$$ and by independence we have $$\text{Var}[S] = \sum_i \text{Var}[c_iX_i] = \sum_i c_i^2 \Omega_i$$ so $$S\sim\mathcal N\left(\sum_i c_i\mu_i, \sum_i c_i^2\Omega_i\right).$$ This applies no matter what the $$c_i$$ are and for any dimension of $$X_i$$.

The above part assumed $$n < \infty$$. If we have a countably infinite number of $$X_i$$ then whether or not the series $$\sum_{i=1}^\infty c_i X_i$$ converges depends on how the $$c_i$$, $$\mu_i$$, and $$\Omega_i$$ evolve and we can use Kolmogrov's three series theorem to understand when this happens.

I interpreted this to mean you wanted the distribution of a linear combination of Gaussians. If you meant a finite mixture of Gaussians then we can work it out in the following way. Let $$f_i$$ be the density of $$X_i$$ and let $$S \sim \sum_{i=1}^n c_i f_i$$ be the mixture distribution. You didn't state that $$c_i \geq 0$$ but I'll assume that so that this is a valid density. Then we have $$\text E[S] = \int s \sum_i c_i f_i(s)\,\text ds = \sum_i c_i \text E[X_i] = \sum_i c_i \mu_i$$ as before, except now this represents a convex combination of the $$\mu_i$$ where that was not guaranteed before. I'll use $$\mu_\text{mix} = \sum_i c_i\mu_i$$ as the mixture mean.

For the variances we need $$\text E[SS^T] = \int ss^T \sum_i c_i f_i(s)\,\text ds = \sum_i c_i \text E[X_iX_i^T]$$ so all together $$\text{Var}[S] = \text E[SS^T] - (\text E S)(\text ES)^T \\ =\sum_i c_i \text E[X_iX_i^T] - \mu_\text{mix}\mu_\text{mix}^T$$ which is more complicated than $$\sum_i c_i^2\Omega_i$$

• Thanks! This was my intuition & I appreciate the quick response.
– CJR
Jul 8 '21 at 19:24
• @CJR for sure! glad this helped
– jld
Jul 8 '21 at 19:30
• @CJR i just updated for the mixture case since it seems like maybe that's what you meant?
– jld
Jul 8 '21 at 20:16
• Thanks for updating the post. I am still but unclear how the last line produces the variance-covariance matrix for the resulting mixture bivariate normal? If you could clarify that a bit more I'd really appreciate it. @jld
– CJR
Jul 8 '21 at 21:01
• @CJR yeah sure. I'm using the result that for a random vector $S$ with mean vector $\mu$ the variance is $\text{Var}[S] = \text E[SS^T] - \mu\mu^T$. We know $\mu$ so the only remaining thing is the first term of $\text E[SS^T]$ and that's just the expected value of the 2x2 matrices $ss^T$ weighted by the mixture density
– jld
Jul 8 '21 at 21:45

Thanks to @jld & @whuber for their answeres. Both were super helpful as I tried to solve this problem. With continued research, I found the post which I'll link below. It has the same info that jld & whuber shared, but it helped me solve it so I wanted to link it here

https://math.stackexchange.com/questions/195911/calculation-of-the-covariance-of-gaussian-mixtures