58
$\begingroup$

I know that if $$X\sim N(\mu_X, \sigma^2_X), \hspace{1em} Y \sim N(\mu_Y, \sigma^2_Y), \hspace{1em} Z = X + Y, \hspace{1em} X,Y \text{ independent}$$ then $$Z \sim N(\mu_X + \mu_Y, \sigma^2_X + \sigma^2_Y).$$

But what would happen if $X,Y$ were not independent? i.e. if $$ \begin{pmatrix}X\\Y\end{pmatrix} \sim N\left( \begin{pmatrix}\mu_X\\\mu_Y\end{pmatrix}, \begin{pmatrix} \sigma^2_X & \sigma_{X,Y} \\ \sigma_{X,Y} & \sigma^2_Y \end{pmatrix} \right). $$

Would this affect how the sum $Z$ is distributed?

$\endgroup$
7
  • 12
    $\begingroup$ Just would like to point out that there are all sorts of joint distributions for $(X,Y)$ other than bivariate normal that still have $X$ and $Y$ marginally normal. And this distinction would make a huge difference on the answers. $\endgroup$
    – user1108
    Dec 18, 2011 at 0:09
  • 3
    $\begingroup$ @G.JayKerns I agree that if $X$ and $Y$ are normal but not necessarily jointly normal, then $X+Y$ can have distribution other than normal. But the OP's statement that "$Z$ is distributed $N(\mu_x + \mu_y, \sigma^2_x + \sigma^2_y)$ if $X$ and $Y$ are independent." is absolutely correct. If $X$ and $Y$ are marginally normal (as the first part of the sentence says) and independent (as per the assumption in the second part of the sentence), then they are also jointly normal. In the OP's question, joint normality is assumed explicitly and so any linear combination of $X$ and $Y$ is normal. $\endgroup$ Dec 18, 2011 at 0:42
  • 6
    $\begingroup$ @Dilip, let me be clear that there is nothing wrong with the question and there is nothing wrong with your answer (+1) (or probability's, either (+1)). I was simply pointing out that if $X$ and $Y$ are dependent then it isn't necessary that they are jointly normal, and it wasn't clear that the OP had considered that possibility. In addition, I am afraid (though I haven't spent a lot of time thinking) that without some other assumptions (like joint normality) the question might even be unanswerable. $\endgroup$
    – user1108
    Dec 18, 2011 at 15:52
  • 8
    $\begingroup$ As @G.JayKerns mentions, of course we can get all sorts of interesting behavior if we consider marginally, but not jointly, distributed normals. Here is a simple example: Let $X$ be standard normal and $\varepsilon = \pm 1$ with probability 1/2 each, independently of $X$. Let $Y = \varepsilon X$. Then $Y$ is also standard normal, but $Z = X+Y$ is exactly equal to zero with probability 1/2 and is equal to $2X$ with probability 1/2. $\endgroup$
    – cardinal
    Dec 18, 2011 at 22:56
  • 5
    $\begingroup$ We can get a whole variety of different behaviors by considering the bivariate copula that is associated with $(X,Y)$ via Sklar's theorem. If we use the Gaussian copula, then we get $(X,Y)$ are jointly normal, and so $Z = X + Y$ is normally distributed. If the copula is not the Gaussian copula, then $X$ and $Y$ are each still marginally distributed as normals, but are not jointly normal and so the sum will not be normally distributed, in general. $\endgroup$
    – cardinal
    Dec 18, 2011 at 23:00

2 Answers 2

42
$\begingroup$

See my comment on probabilityislogic's answer to this question. Here, $$ \begin{align*} X + Y &\sim N(\mu_X + \mu_Y,\; \sigma_X^2 + \sigma_Y^2 + 2\sigma_{X,Y})\\ aX + bY &\sim N(a\mu_X + b\mu_Y,\; a^2\sigma_X^2 + b^2\sigma_Y^2 + 2ab\sigma_{X,Y}) \end{align*} $$ where $\sigma_{X,Y}$ is the covariance of $X$ and $Y$. Nobody writes the off-diagonal entries in the covariance matrix as $\sigma_{xy}^2$ as you have done. The off-diagonal entries are covariances which can be negative.

$\endgroup$
0
43
$\begingroup$

@dilip's answer is sufficient, but I just thought I'd add some details on how you get to the result. We can use the method of characteristic functions. For any $d$-dimensional multivariate normal distribution $X\sim N_{d}(\mu,\Sigma)$ where $\mu=(\mu_1,\dots,\mu_d)^T$ and $\Sigma_{jk}=cov(X_j,X_k)\;\;j,k=1,\dots,d$, the characteristic function is given by:

$$\varphi_{X}({\bf{t}})=E\left[\exp(i{\bf{t}}^TX)\right]=\exp\left(i{\bf{t}}^T\mu-\frac{1}{2}{\bf{t}}^T\Sigma{\bf{t}}\right)$$ $$=\exp\left(i\sum_{j=1}^{d}t_j\mu_j-\frac{1}{2}\sum_{j=1}^{d}\sum_{k=1}^{d}t_jt_k\Sigma_{jk}\right)$$

For a one-dimensional normal variable $Y\sim N_1(\mu_Y,\sigma_Y^2)$ we get:

$$\varphi_Y(t)=\exp\left(it\mu_Y-\frac{1}{2}t^2\sigma_Y^2\right)$$

Now, suppose we define a new random variable $Z={\bf{a}}^TX=\sum_{j=1}^{d}a_jX_j$. For your case, we have $d=2$ and $a_1=a_2=1$. The characteristic function for $Z$ is the basically the same as that for $X$.

$$\varphi_{Z}(t)=E\left[\exp(itZ)\right]=E\left[\exp(it{\bf{a}}^TX)\right]=\varphi_{X}(t{\bf{a}})$$ $$=\exp\left(it\sum_{j=1}^{d}a_j\mu_j-\frac{1}{2}t^2\sum_{j=1}^{d}\sum_{k=1}^{d}a_ja_k\Sigma_{jk}\right)$$

If we compare this characteristic function with the characteristic function $\varphi_Y(t)$ we see that they are the same, but with $\mu_Y$ being replaced by $\mu_Z=\sum_{j=1}^{d}a_j\mu_j$ and with $\sigma_Y^2$ being replaced by $\sigma^2_Z=\sum_{j=1}^{d}\sum_{k=1}^{d}a_ja_k\Sigma_{jk}$. Hence because the characteristic function of $Z$ is equivalent to the characteristic function of $Y$, the distributions must also be equal. Hence $Z$ is normally distributed. We can simplify the expression for the variance by noting that $\Sigma_{jk}=\Sigma_{kj}$ and we get:

$$\sigma^2_Z=\sum_{j=1}^{d}a_j^2\Sigma_{jj}+2\sum_{j=2}^{d}\sum_{k=1}^{j-1}a_ja_k\Sigma_{jk}$$

This is also the general formula for the variance of a linear combination of any set of random variables, independent or not, normal or not, where $\Sigma_{jj}=var(X_j)$ and $\Sigma_{jk}=cov(X_j,X_k)$. Now if we specialise to $d=2$ and $a_1=a_2=1$, the above formula becomes:

$$\sigma^2_Z=\sum_{j=1}^{2}(1)^2\Sigma_{jj}+2\sum_{j=2}^{2}\sum_{k=1}^{j-1}(1)(1)\Sigma_{jk}=\Sigma_{11}+\Sigma_{22}+2\Sigma_{21}$$

$\endgroup$
1
  • 3
    $\begingroup$ +1 Thanks for taking the time to write out the details. Can this question be made part of the FAQ? $\endgroup$ Dec 18, 2011 at 13:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.