12
$\begingroup$

Suppose we have two vectors of random variables, both are normal, i.e., $X \sim N(\mu_X, \Sigma_X)$ and $Y \sim N(\mu_Y, \Sigma_Y)$. We are interested in the distribution of their linear combination $Z = A X + B Y + C$, where $A$ and $B$ are matrices, $C$ is a vector. If $X$ and $Y$ are independent, $Z \sim N(A \mu_X + B \mu_Y + C, A \Sigma_X A^T + B \Sigma_Y B^T)$. The question is in the dependent case, assuming that we know the correlation of any pair $(X_i, Y_i)$. Thank you.

Best wishes, Ivan

$\endgroup$
7
$\begingroup$

In that case, you have to write (with hopefully clear notations) $$ \left(\begin{matrix}X\\Y \end{matrix}\right) \sim \mathcal{N}\left[ \left(\begin{matrix}\mu_X\\\mu_Y\end{matrix}\right), \Sigma_{X,Y} \right] $$ (edited: assuming joint normality of $(X,Y)$) Then $$ AX+BY=\left(\begin{matrix}A& B \end{matrix}\right) \left(\begin{matrix}X\\Y \end{matrix}\right) $$ and $$ AX+BY+C \sim \mathcal{N}\left[ \left(\begin{matrix}A& B \end{matrix}\right) \left(\begin{matrix}\mu_X\\\mu_Y\end{matrix}\right) + C, \left(\begin{matrix}A & B \end{matrix}\right)\Sigma_{X,Y} \left(\begin{matrix}A^T \\ B^T \end{matrix}\right)\right] $$ i.e. $$ AX+BY+C \sim \mathcal{N}\left[A\mu_X + B\mu_Y +C, A\Sigma_{XX}A^T+B\Sigma_{XY}^TA^T+A\Sigma_{XY}B^T+B\Sigma_{YY}B^T \right] $$

$\endgroup$
  • 3
    $\begingroup$ In case it's overlooked, please note that the comment thread to another reply indicates (a) these covariance calculations are fine (understanding that they involve a natural but unstated block matrix notation) but (b) we cannot validly conclude that the linear combinations are normally distributed until we make an additional assumption; namely, that $X$ and $Y$ have a joint multivariate normal distribution. $\endgroup$ – whuber Feb 15 '12 at 16:41
  • 2
    $\begingroup$ Could you explain how you got from $B\Sigma_{XY}^TA^T+A\Sigma_{XY}B^T$ to $2A\Sigma_{XY}B^T$ in the last line? I would have thought that $$B\Sigma_{XY}^TA^T+A\Sigma_{XY}B^T = (A\Sigma_{XY}B^T)^T + A\Sigma_{XY}B^T$$ and the result does not simplify further. Here $\Sigma_{XY}$ is not a symmetric matrix since its $(i,j)$-th element is $\text{cov}(X_i,Y_j)$ while its $(j,i)$-th element is $\text{cov}(X_j,Y_i)$, and there is no reason why these covariances must be equal. $\endgroup$ – Dilip Sarwate Feb 15 '12 at 20:10
  • 1
    $\begingroup$ @DilipSarwate: (+1) you are right, in the general case, there is no reason for these two terms to be equal. $\endgroup$ – Xi'an Feb 15 '12 at 20:19
3
$\begingroup$

Your question does not have a unique answer as currently posed unless you assume that $X$and$Y$ are jointly normally distributed with covariance top right block $\Sigma_{XY}$. i think you do mean this because you say you have each covariance between X and Y. In this case we can write $W=(X^T,Y^T)^T$ which is also multivariate normal. then $Z$ is given in terms of $W$ as:

$$Z=(A,B)W+C$$

Then you use your usual formula for linear combination. Note that the mean is unchanged but the covariance matrix has two extra terms added $A\Sigma_{XY}B^T+B\Sigma_{XY}^TA^T$

$\endgroup$
  • $\begingroup$ Thanks for pointing out this issue, in fact, I didn't even think about it, but it seems that the variables are indeed can be viewed, in my case, as jointly normally distributed, even if their components are correlated. $\endgroup$ – Ivan Feb 15 '12 at 12:45
  • $\begingroup$ I agree that the question cannot be solved as posed. It can be solved in a straightforward manner if one assumes, as @Xi'an's answer does, that $X$ and $Y$ are jointly normally distributed. It could be solvable, presumably with more difficulty, if the joint distribution was specified as something other than joint normal. But just knowing $\text{cov}(X_i,Y_j)$ for all $i, j$, does not mean that $W = (X^T,Y^T)^T$ is multivariate normal. Any two random variables with finite variances have a covariance. Covariance is not defined only for normal or jointly normal random variables. $\endgroup$ – Dilip Sarwate Feb 15 '12 at 13:19
  • $\begingroup$ In my case, X and Y are jointly normal, I will try to explain why, please correct me if I am wrong. Suppose there is a set of independent univariate normal r.v.'s. Each element of X and Y is an arbitrary linear combination of these univariate variables from the set. Therefore, since the initial variables are independent and only linear transformations are involved, the resulting vectors X, Y, and Z are all multivariate normal r.v.'s. It follows the definition of a multivariate normal r.v., where $a^T X$ should be a univariate normal r.v. for any vector $a$. Does it make sense? $\endgroup$ – Ivan Feb 15 '12 at 14:05
  • 1
    $\begingroup$ @Ivan Your explanation makes sense but the complaint is about the statement "Suppose we have two vectors of random variables, both are normal, i.e., $X\sim N(\mu_X,\Sigma_X)$ and $Y\sim N(\mu_Y,\Sigma_Y)$" which does not mean that $X$ and $Y$ are jointly normal. Nor does saying that "we know the correlation of any pair $(X_i,Y_i)$" mean that $X_i$ and $Y_i$ are jointly normal even though, as you correctly state, $X\sim N(\mu_X,\Sigma_X)$ implies that $X_i$ is normal (and similarly for $Y_i$.) Univariate normality does not imply joint normality. See reference below. $\endgroup$ – Dilip Sarwate Feb 15 '12 at 14:25
  • $\begingroup$ @Ivan See the discussion following this question $\endgroup$ – Dilip Sarwate Feb 15 '12 at 14:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.