# Are a sum and a product of two covariance matrices also a covariance matrix?

Suppose I have covariance matrices $X$ and $Y$. Which of these options are then also covariance matrices?

1. $X+Y$
2. $X^2$
3. $XY$

I have a bit of trouble understanding what exactly is needed for something to be a covariance matrix. I suppose it is meant that for instance if $X=\operatorname{cov}(X_1,X_2)$, and $Y=\operatorname{cov}(Y_1,Y_2)$ that for 1 to hold true we should have that $\operatorname{cov}(X_1,X_2) + \operatorname{cov}(Y_1,Y_2) = \operatorname{cov}(Z_1, Z_2)$, where $Z_1$ and $Z_2$ are some other random variables. However, I can't see why that would hold true for any of the three options. Any insight would be apprciated.

### Background

A covariance matrix $$\mathbb{A}$$ for a vector of random variables $$X=(X_1, X_2, \ldots, X_n)^\prime$$ embodies a procedure to compute the variance of any linear combination of those random variables. The rule is that for any vector of coefficients $$\lambda = (\lambda_1, \ldots, \lambda_n)$$,

$$\operatorname{Var}(\lambda X) = \lambda \mathbb{A} \lambda ^\prime.\tag{1}$$

In other words, the rules of matrix multiplication describe the rules of variances.

Two properties of $$\mathbb{A}$$ are immediate and obvious:

1. Because variances are expectations of squared values, they can never be negative. Thus, for all vectors $$\lambda$$, $$0 \le \operatorname{Var}(\lambda X) = \lambda \mathbb{A} \lambda ^\prime.$$ Covariance matrices must be non-negative-definite.

2. Variances are just numbers--or, if you read the matrix formulas literally, they are $$1\times 1$$ matrices. Thus, they do not change when you transpose them. Transposing $$(1)$$ gives $$\lambda \mathbb{A} \lambda ^\prime = \operatorname{Var}(\lambda X) = \operatorname{Var}(\lambda X) ^\prime = \left(\lambda \mathbb{A} \lambda ^\prime\right)^\prime = \lambda \mathbb{A}^\prime \lambda ^\prime.$$ Since this holds for all $$\lambda$$, $$\mathbb{A}$$ must equal its transpose $$\mathbb{A}^\prime$$: covariance matrices must be symmetric.

The deeper result is that any non-negative-definite symmetric matrix $$\mathbb{A}$$ is a covariance matrix. This means there actually is some vector-valued random variable $$X$$ with $$\mathbb{A}$$ as its covariance. We may demonstrate this by explicitly constructing $$X$$. One way is to notice that the (multivariate) density function $$f(x_1,\ldots, x_n)$$ with the property $$\log(f) \propto -\frac{1}{2} (x_1,\ldots,x_n)\mathbb{A}^{-1}(x_1,\ldots,x_n)^\prime$$ has $$\mathbb{A}$$ for its covariance. (Some delicacy is needed when $$\mathbb{A}$$ is not invertible--but that's just a technical detail.)

### Solutions

Let $$\mathbb{X}$$ and $$\mathbb{Y}$$ be covariance matrices. Obviously they are square; and if their sum is to make any sense they must have the same dimensions. We need only check the two properties.

1. The sum.

• Symmetry $$(\mathbb{X}+\mathbb{Y})^\prime = \mathbb{X}^\prime + \mathbb{Y}^\prime = (\mathbb{X} + \mathbb{Y})$$ shows the sum is symmetric.
• Non-negative definiteness. Let $$\lambda$$ be any vector. Then $$\lambda(\mathbb{X}+\mathbb{Y})\lambda^\prime = \lambda \mathbb{X}\lambda^\prime + \lambda \mathbb{Y}\lambda^\prime \ge 0 + 0 = 0$$ proves the point using basic properties of matrix multiplication.
2. I leave this as an exercise.

3. This one is tricky. One method I use to think through challenging matrix problems is to do some calculations with $$2\times 2$$ matrices. There are some common, familiar covariance matrices of this size, such as $$\pmatrix{a & b \\ b & a}$$ with $$a^2 \ge b^2$$ and $$a \ge 0$$. The concern is that $$\mathbb{XY}$$ might not be definite: that is, could it produce a negative value when computing a variance? If it will, then we had better have some negative coefficients in the matrix. That suggests considering $$\mathbb{X} = \pmatrix{a & -1 \\ -1 & a}$$ for $$a \ge 1$$. To get something interesting, we might gravitate initially to matrices $$\mathbb{Y}$$ with different-looking structures. Diagonal matrices come to mind, such as $$\mathbb{Y} = \pmatrix{b & 0 \\ 0 & 1}$$ with $$b\ge 0$$. (Notice how we may freely pick some of the coefficients, such as $$-1$$ and $$1$$, because we can rescale all the entries in any covariance matrix without changing its fundamental properties. This simplifies the search for interesting examples.)

I leave it to you to compute $$\mathbb{XY}$$ and test whether it always is a covariance matrix for any allowable values of $$a$$ and $$b$$.

A real matrix is a covariance matrix if and only if it is symmetric positive semi-definite.

Hints:

1) If $X$ and $Y$ are symmetric, is $X+Y$ symmetric? If $z^TX z \ge 0$ for any $z$ and $z^TY z \ge 0$ for any $z$, what can you conclude about $z^T(X+Y)z$?

2) If $X$ is symmetric, is $X^2$ symmetric? If the eigenvalues of $X$ are non-negative, what can you conclude about the eigenvalues of $X^2$?

3) If $X$ and $Y$ are symmetric, can you conclude that $XY$ is symmetric, or can you find a counter-example?