2
$\begingroup$

Let X and Y be random vectors of same dimension.

Let var(X) be the covariance matrix of X; var(Y) the correspondent matrix; cov(X,Y) the matrix where the coordinate (i,j) is cov(x_i, y_j)

I saw the affirmation that var(X+Y) is var(X)+ var(Y) + 2 cov(X,Y), and have been trying to prove it.

For positions (i,i), the fact seems simple enough. Take, however, the position (1,2). Making A=X+Y, that position it is supposed to be

$$ \text{cov}(a_1,a_2)$$ $$ E [(x_1+y_1-\mu_{x_1}-\mu_{y_1}) (x_2+y_2-\mu_{x_2}-\mu_{y_2})]$$ $$ E (x_1-\mu_{x_1} + y_1-\mu_{y_1}) (x_2-\mu_{x_2} + y_2-\mu_{y_2}) $$ $$ E (x_1-\mu_{x_1})(x_2-\mu_{x_2})+ E(x_1-\mu_{x_1})(y_2-\mu_{y_2})+ E(y_1-\mu_{y_1}) (x_2-\mu_{x_2}) + E(y_1-\mu_{y_1})(y_2-\mu_{y_2})$$ $$ \text{cov}(x_1,x_2)+ \text{cov}(x_1,y_2) + \text{cov}(x_2,y_1) + \text{cov}(y_1,y_2)$$

$$ \text{cov}(x_1,x_2)+ \text{cov}(y_1,y_2) + \text{cov}(x_1,y_2) + \text{cov}(x_2,y_1) $$

The first two terms come from the covariance matrixes of x and y, but the last two seem to fit 2*cov(x,y) only if that last matrix is symetric (and there does not seem to be any reason for this to be the case...)

$\endgroup$
0

2 Answers 2

3
$\begingroup$

The variance covariance matrix is defined by \begin{align} \mathbb E[(X+Y-\mu_X-\mu_Y)&(X+Y-\mu_X-\mu_Y)^\top]\\ &= \mathbb E[(X-\mu_X+Y-\mu_Y)(X-\mu_X+Y-\mu_Y)^\top] \\ &= \mathbb E[(X-\mu_X)(X-\mu_X)^\top]\\ &\quad + \mathbb E[(Y-\mu_Y)(Y-\mu_Y)^\top]\\ &\quad + \mathbb E[(Y-\mu_Y)(X-\mu_X)^\top]\\ &\quad + \mathbb E[(X-\mu_X)(Y-\mu_Y)^\top]\\ &= \mathbb E[(X-\mu_X)(X-\mu_X)^\top]\\ &\quad + \mathbb E[(Y-\mu_Y)(Y-\mu_Y)^\top]\\ &\quad + 2\mathbb E[(Y-\mu_Y)(X-\mu_X)^\top]\qquad\text{[symmetry]}\\ &= \operatorname{Var}(X)+\operatorname{Var}(Y)+2\operatorname{Cov}(X,Y) \end{align}

Edit:

The last step is wrong, because in this question X and Y are random vectors, not random variables.

The covariance for two random variables is "symmetric", because $$\operatorname{Cov}(x,y):=\mathbb E[(x-\mathbb E(x))(y-\mathbb E(y))]=\mathbb E[xy]-\mathbb E[x] \mathbb E[y]=\operatorname{Cov}(y,x)$$ However, the cross covariance matrix for two random vectors is not necessarily symmetric, $$ \operatorname{Cov}(X,Y):=\mathbb E[(X-\mathbb E(X))(Y-\mathbb E(Y))^\top]=\mathbb E[XY^\top]+\mathbb E[X]\mathbb E[Y^\top]\\ $$ Note that we do have this relationship, $$ \operatorname{Cov}(X,Y):=\mathbb E[(X-\mathbb E(X))(Y-\mathbb E(Y))^\top]\\ =\mathbb E[((Y-\mathbb E(Y))(X-\mathbb E(X))^\top)^\top]\\ =(\mathbb E[(Y-\mathbb E(Y))(X-\mathbb E(X))^\top])^\top\\ =\operatorname{Cov}(Y,X)^\top\\ $$ Therefore, \begin{align} &\mathbb E[(X-\mu_X)(X-\mu_X)^\top]\\ &\quad + \mathbb E[(Y-\mu_Y)(Y-\mu_Y)^\top]\\ &\quad + \mathbb E[(Y-\mu_Y)(X-\mu_X)^\top]\\ &\quad + \mathbb E[(X-\mu_X)(Y-\mu_Y)^\top]\\ &= \operatorname{Var}(X)+\operatorname{Var}(Y)+\operatorname{Cov}(Y,X)+\operatorname{Cov}(X,Y)\\ &= \operatorname{Var}(X)+\operatorname{Var}(Y)+\operatorname{Cov}(X,Y)^\top+\operatorname{Cov}(X,Y).\\ \end{align}

$\endgroup$
4
  • $\begingroup$ why is that matrix symmetrical, though? $\endgroup$
    – josinalvo
    Oct 19, 2020 at 13:24
  • $\begingroup$ Hint: take one element at a time. $\endgroup$
    – Xi'an
    Oct 19, 2020 at 13:39
  • $\begingroup$ Sorry, I seem to be really stuck :P In the matrix E[(Y-mu_Y)(X-mu_X)^t] the position (1,2) is E(Y1-mu_y1)(X2-mu_x2) = cov(Y_1,X_2), right? $\endgroup$
    – josinalvo
    Oct 19, 2020 at 13:45
  • $\begingroup$ While the position (2,1) is E(Y2-mu_y2)(X1-mu_x1) = cov(Y_2,X_1), right? $\endgroup$
    – josinalvo
    Oct 19, 2020 at 13:46
0
$\begingroup$

Your notation causes the confusion. If you use two subscripts for the individual elements, it becomes much simpler to show. Assuming the means are zero ( for ease of notation ):

$E(x_{(1,1)} + y_{(1,2)})^2 = $

$E(x_{(1,1)}^2) + E(x_{(1,1)}y_{(1,2)}) + E(y_{(1,2)}x_{(1,1)}) + E(y_{1,2}^2) = $

$ Var(x_{(1,1)}) + Var(y_{(1,2)}) + 2 \times cov(y_{(1,2)}, x_{(1,1)}) $

$\endgroup$
4
  • $\begingroup$ I must have some fundamental misunderstanding... X and Y are in Rn, right? Why would I use two indices? $\endgroup$
    – josinalvo
    Oct 19, 2020 at 13:25
  • 1
    $\begingroup$ Hi: Maybe my answer was more confusing than helpful. You really don't need two indices. By (1,1), I was referrring to the first x element in the $X$ vector and, by (1,2), I was referring to the second element in the $Y$ vector. So, when I wrote above, it is for two specific elements of each vector and not the whole vector. But the result of it goes in the [first row, second column] and in the [second row, first column] positions of the covariance matrix. So that was why I gave them the two subscripts. I hope it clarifies the confusion. $\endgroup$
    – mlofton
    Oct 19, 2020 at 19:31
  • $\begingroup$ Hi: One more thing. Note that, if you do the same thing for all the elements, you will obtain an $n \times n $ covariance matrix so yes, it's consistent with both vectors being in $R^{n}$. I just did it for two specific elements of the $R^n$ vectors. $\endgroup$
    – mlofton
    Oct 19, 2020 at 19:36
  • $\begingroup$ Note that Xian did it all at once very nicely so you're probably better off considering his answer. The symmetry ( in the scalar case) is due to the fact that cov(x,y) = E(xy) - E(x)E(y). This formula shows that the order is irrelevant which gives you the symmetry. $\endgroup$
    – mlofton
    Oct 19, 2020 at 19:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.