19
$\begingroup$

Suppose we have two vectors of random variables, both are normal, i.e., $X \sim N(\mu_X, \Sigma_X)$ and $Y \sim N(\mu_Y, \Sigma_Y)$. We are interested in the distribution of their linear combination $Z = A X + B Y + C$, where $A$ and $B$ are matrices, $C$ is a vector. If $X$ and $Y$ are independent, $Z \sim N(A \mu_X + B \mu_Y + C, A \Sigma_X A^T + B \Sigma_Y B^T)$. The question is in the dependent case, assuming that we know the correlation of any pair $(X_i, Y_i)$. Thank you.

Best wishes, Ivan

Improve this question
$\endgroup$

2 Answers 2

Reset to default
16
$\begingroup$

In that case, you have to write (with hopefully clear notations) $$ \left(\begin{matrix}X\\Y \end{matrix}\right) \sim \mathcal{N}\left[ \left(\begin{matrix}\mu_X\\\mu_Y\end{matrix}\right), \Sigma_{X,Y} \right] $$ (edited: assuming joint normality of $(X,Y)$) Then $$ AX+BY=\left(\begin{matrix}A& B \end{matrix}\right) \left(\begin{matrix}X\\Y \end{matrix}\right) $$ and $$ AX+BY+C \sim \mathcal{N}\left[ \left(\begin{matrix}A& B \end{matrix}\right) \left(\begin{matrix}\mu_X\\\mu_Y\end{matrix}\right) + C, \left(\begin{matrix}A & B \end{matrix}\right)\Sigma_{X,Y} \left(\begin{matrix}A^T \\ B^T \end{matrix}\right)\right] $$ i.e. $$ AX+BY+C \sim \mathcal{N}\left[A\mu_X + B\mu_Y +C, A\Sigma_{XX}A^T+B\Sigma_{XY}^TA^T+A\Sigma_{XY}B^T+B\Sigma_{YY}B^T \right] $$

Improve this answer
$\endgroup$
3
  • 6
    $\begingroup$ In case it's overlooked, please note that the comment thread to another reply indicates (a) these covariance calculations are fine (understanding that they involve a natural but unstated block matrix notation) but (b) we cannot validly conclude that the linear combinations are normally distributed until we make an additional assumption; namely, that $X$ and $Y$ have a joint multivariate normal distribution. $\endgroup$
    – whuber ♦
    Commented Feb 15, 2012 at 16:41
  • 4
    $\begingroup$ Could you explain how you got from $B\Sigma_{XY}^TA^T+A\Sigma_{XY}B^T$ to $2A\Sigma_{XY}B^T$ in the last line? I would have thought that $$B\Sigma_{XY}^TA^T+A\Sigma_{XY}B^T = (A\Sigma_{XY}B^T)^T + A\Sigma_{XY}B^T$$ and the result does not simplify further. Here $\Sigma_{XY}$ is not a symmetric matrix since its $(i,j)$-th element is $\text{cov}(X_i,Y_j)$ while its $(j,i)$-th element is $\text{cov}(X_j,Y_i)$, and there is no reason why these covariances must be equal. $\endgroup$
    – Dilip Sarwate
    Commented Feb 15, 2012 at 20:10
  • 2
    $\begingroup$ @DilipSarwate: (+1) you are right, in the general case, there is no reason for these two terms to be equal. $\endgroup$
    – Xi'an
    Commented Feb 15, 2012 at 20:19
4
$\begingroup$

Your question does not have a unique answer as currently posed unless you assume that $X$and$Y$ are jointly normally distributed with covariance top right block $\Sigma_{XY}$. i think you do mean this because you say you have each covariance between X and Y. In this case we can write $W=(X^T,Y^T)^T$ which is also multivariate normal. then $Z$ is given in terms of $W$ as:

$$Z=(A,B)W+C$$

Then you use your usual formula for linear combination. Note that the mean is unchanged but the covariance matrix has two extra terms added $A\Sigma_{XY}B^T+B\Sigma_{XY}^TA^T$

Improve this answer
$\endgroup$
7
  • $\begingroup$ Thanks for pointing out this issue, in fact, I didn't even think about it, but it seems that the variables are indeed can be viewed, in my case, as jointly normally distributed, even if their components are correlated. $\endgroup$
    – Ivan
    Commented Feb 15, 2012 at 12:45
  • 1
    $\begingroup$ I agree that the question cannot be solved as posed. It can be solved in a straightforward manner if one assumes, as @Xi'an's answer does, that $X$ and $Y$ are jointly normally distributed. It could be solvable, presumably with more difficulty, if the joint distribution was specified as something other than joint normal. But just knowing $\text{cov}(X_i,Y_j)$ for all $i, j$, does not mean that $W = (X^T,Y^T)^T$ is multivariate normal. Any two random variables with finite variances have a covariance. Covariance is not defined only for normal or jointly normal random variables. $\endgroup$
    – Dilip Sarwate
    Commented Feb 15, 2012 at 13:19
  • $\begingroup$ In my case, X and Y are jointly normal, I will try to explain why, please correct me if I am wrong. Suppose there is a set of independent univariate normal r.v.'s. Each element of X and Y is an arbitrary linear combination of these univariate variables from the set. Therefore, since the initial variables are independent and only linear transformations are involved, the resulting vectors X, Y, and Z are all multivariate normal r.v.'s. It follows the definition of a multivariate normal r.v., where $a^T X$ should be a univariate normal r.v. for any vector $a$. Does it make sense? $\endgroup$
    – Ivan
    Commented Feb 15, 2012 at 14:05
  • 1
    $\begingroup$ @Ivan Your explanation makes sense but the complaint is about the statement "Suppose we have two vectors of random variables, both are normal, i.e., $X\sim N(\mu_X,\Sigma_X)$ and $Y\sim N(\mu_Y,\Sigma_Y)$" which does not mean that $X$ and $Y$ are jointly normal. Nor does saying that "we know the correlation of any pair $(X_i,Y_i)$" mean that $X_i$ and $Y_i$ are jointly normal even though, as you correctly state, $X\sim N(\mu_X,\Sigma_X)$ implies that $X_i$ is normal (and similarly for $Y_i$.) Univariate normality does not imply joint normality. See reference below. $\endgroup$
    – Dilip Sarwate
    Commented Feb 15, 2012 at 14:25
  • $\begingroup$ @Ivan See the discussion following this question $\endgroup$
    – Dilip Sarwate
    Commented Feb 15, 2012 at 14:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.