# Linear combination of two dependent multivariate normal random variables

Suppose we have two vectors of random variables, both are normal, i.e., $X \sim N(\mu_X, \Sigma_X)$ and $Y \sim N(\mu_Y, \Sigma_Y)$. We are interested in the distribution of their linear combination $Z = A X + B Y + C$, where $A$ and $B$ are matrices, $C$ is a vector. If $X$ and $Y$ are independent, $Z \sim N(A \mu_X + B \mu_Y + C, A \Sigma_X A^T + B \Sigma_Y B^T)$. The question is in the dependent case, assuming that we know the correlation of any pair $(X_i, Y_i)$. Thank you.

Best wishes, Ivan

In that case, you have to write (with hopefully clear notations) $$\left(\begin{matrix}X\\Y \end{matrix}\right) \sim \mathcal{N}\left[ \left(\begin{matrix}\mu_X\\\mu_Y\end{matrix}\right), \Sigma_{X,Y} \right]$$ (edited: assuming joint normality of $(X,Y)$) Then $$AX+BY=\left(\begin{matrix}A& B \end{matrix}\right) \left(\begin{matrix}X\\Y \end{matrix}\right)$$ and $$AX+BY+C \sim \mathcal{N}\left[ \left(\begin{matrix}A& B \end{matrix}\right) \left(\begin{matrix}\mu_X\\\mu_Y\end{matrix}\right) + C, \left(\begin{matrix}A & B \end{matrix}\right)\Sigma_{X,Y} \left(\begin{matrix}A^T \\ B^T \end{matrix}\right)\right]$$ i.e. $$AX+BY+C \sim \mathcal{N}\left[A\mu_X + B\mu_Y +C, A\Sigma_{XX}A^T+B\Sigma_{XY}^TA^T+A\Sigma_{XY}B^T+B\Sigma_{YY}B^T \right]$$

• In case it's overlooked, please note that the comment thread to another reply indicates (a) these covariance calculations are fine (understanding that they involve a natural but unstated block matrix notation) but (b) we cannot validly conclude that the linear combinations are normally distributed until we make an additional assumption; namely, that $X$ and $Y$ have a joint multivariate normal distribution. – whuber Feb 15 '12 at 16:41
• Could you explain how you got from $B\Sigma_{XY}^TA^T+A\Sigma_{XY}B^T$ to $2A\Sigma_{XY}B^T$ in the last line? I would have thought that $$B\Sigma_{XY}^TA^T+A\Sigma_{XY}B^T = (A\Sigma_{XY}B^T)^T + A\Sigma_{XY}B^T$$ and the result does not simplify further. Here $\Sigma_{XY}$ is not a symmetric matrix since its $(i,j)$-th element is $\text{cov}(X_i,Y_j)$ while its $(j,i)$-th element is $\text{cov}(X_j,Y_i)$, and there is no reason why these covariances must be equal. – Dilip Sarwate Feb 15 '12 at 20:10
• @DilipSarwate: (+1) you are right, in the general case, there is no reason for these two terms to be equal. – Xi'an Feb 15 '12 at 20:19

Your question does not have a unique answer as currently posed unless you assume that $X$and$Y$ are jointly normally distributed with covariance top right block $\Sigma_{XY}$. i think you do mean this because you say you have each covariance between X and Y. In this case we can write $W=(X^T,Y^T)^T$ which is also multivariate normal. then $Z$ is given in terms of $W$ as:

$$Z=(A,B)W+C$$

Then you use your usual formula for linear combination. Note that the mean is unchanged but the covariance matrix has two extra terms added $A\Sigma_{XY}B^T+B\Sigma_{XY}^TA^T$

• Thanks for pointing out this issue, in fact, I didn't even think about it, but it seems that the variables are indeed can be viewed, in my case, as jointly normally distributed, even if their components are correlated. – Ivan Feb 15 '12 at 12:45
• I agree that the question cannot be solved as posed. It can be solved in a straightforward manner if one assumes, as @Xi'an's answer does, that $X$ and $Y$ are jointly normally distributed. It could be solvable, presumably with more difficulty, if the joint distribution was specified as something other than joint normal. But just knowing $\text{cov}(X_i,Y_j)$ for all $i, j$, does not mean that $W = (X^T,Y^T)^T$ is multivariate normal. Any two random variables with finite variances have a covariance. Covariance is not defined only for normal or jointly normal random variables. – Dilip Sarwate Feb 15 '12 at 13:19
• In my case, X and Y are jointly normal, I will try to explain why, please correct me if I am wrong. Suppose there is a set of independent univariate normal r.v.'s. Each element of X and Y is an arbitrary linear combination of these univariate variables from the set. Therefore, since the initial variables are independent and only linear transformations are involved, the resulting vectors X, Y, and Z are all multivariate normal r.v.'s. It follows the definition of a multivariate normal r.v., where $a^T X$ should be a univariate normal r.v. for any vector $a$. Does it make sense? – Ivan Feb 15 '12 at 14:05
• @Ivan Your explanation makes sense but the complaint is about the statement "Suppose we have two vectors of random variables, both are normal, i.e., $X\sim N(\mu_X,\Sigma_X)$ and $Y\sim N(\mu_Y,\Sigma_Y)$" which does not mean that $X$ and $Y$ are jointly normal. Nor does saying that "we know the correlation of any pair $(X_i,Y_i)$" mean that $X_i$ and $Y_i$ are jointly normal even though, as you correctly state, $X\sim N(\mu_X,\Sigma_X)$ implies that $X_i$ is normal (and similarly for $Y_i$.) Univariate normality does not imply joint normality. See reference below. – Dilip Sarwate Feb 15 '12 at 14:25
• @Ivan See the discussion following this question – Dilip Sarwate Feb 15 '12 at 14:27