# Why cov(AX)=A cov(X) A'

I cannot verify the following theorem. Maybe I am doing something wrong, but I don't know what?! Additionally, I'm not sure about the meaning of a constant matrix in the theorem.

Theorem: cov(AX)=Acov(X)A' if A is a constant matrix. (A' means transpose of A, i.e., A^T)

Example:

A (10x5):

0.8727    0.7145    0.9147    0.6370    0.9760
0.8241    0.6210    0.0735    0.7318    0.1231
0.8609    0.7410    0.9865    0.6090    0.7693
0.1135    0.9563    0.2369    0.5369    0.9876
0.7349    0.4313    0.9650    0.1657    0.4418
0.7872    0.6589    0.4490    0.5974    0.7669
0.4247    0.6411    0.9335    0.5238    0.9153
0.2126    0.6212    0.6423    0.3651    0.9752
0.5092    0.6179    0.3732    0.9569    0.1673
0.8009    0.4691    0.4420    0.6289    0.3132

X (5x5):

0.6635    0.6323    0.5883    0.9362    0.0998
0.8298    0.3788    0.4760    0.8170    0.3296
0.8553    0.6836    0.4701    0.5723    0.6803
0.0065    0.8085    0.1542    0.5780    0.7653
0.6389    0.9297    0.1418    0.0438    0.0886

AX (10x5):

2.5820    2.8701    1.5202    2.3352    1.5188
1.2083    1.5127    0.9452    1.7493    0.9079
2.5253    2.7069    1.5259    2.3616    1.5355
1.7059    1.9482    0.8562    1.3768    0.9861
1.9542    1.8325    1.1795    1.7078    1.0380
1.9469    2.2502    1.1887    1.9111    1.1263
2.2004    2.4239    1.2045    1.7985    1.3707
1.8312    2.0105    0.9173    1.3278    1.0287
1.2828    1.7403    0.9404    1.7555    1.2554
1.5029    1.7859    1.0436    1.7633    1.0443

cov(X) 5x5:

0.1190   -0.0345    0.0457    0.0139   -0.0482
-0.0345    0.0429   -0.0316   -0.0557    0.0015
0.0457   -0.0316    0.0419    0.0563   -0.0132
0.0139   -0.0557    0.0563    0.1175    0.0102
-0.0482    0.0015   -0.0132    0.0102    0.1009

cov(AX) (5x5):

0.2227    0.1984    0.0988    0.1011    0.0827
0.1984    0.1960    0.0916    0.1063    0.0874
0.0988    0.0916    0.0576    0.0745    0.0455
0.1011    0.1063    0.0745    0.1144    0.0618
0.0827    0.0874    0.0455    0.0618    0.0510

but A*cov(X)*A' (10x10):

0.2195     0.12119     0.20872    0.077985     0.15076     0.16413     0.17298     0.11964     0.14558     0.15641
0.12119     0.09362     0.12556  -0.0017266     0.10372     0.08966    0.081364    0.034694    0.099912     0.11002
0.20872     0.12556     0.20256    0.057506     0.15142     0.15574     0.15915     0.10193     0.14583      0.1577
0.077985  -0.0017266    0.057506     0.11155    0.015719    0.059279    0.089875    0.099947    0.026385    0.015011
0.15076     0.10372     0.15142    0.015719     0.12378     0.11292     0.10434    0.055612     0.10669     0.12511
0.16413     0.08966     0.15574    0.059279     0.11292     0.12295     0.12914    0.090107     0.10648     0.11599
0.17298    0.081364     0.15915    0.089875     0.10434     0.12914     0.14732     0.11382     0.11134     0.11072
0.11964    0.034694     0.10193    0.099947    0.055612    0.090107     0.11382     0.10483    0.061491    0.057124
0.14558    0.099912     0.14583    0.026385     0.10669     0.10648     0.11134    0.061491     0.12616     0.12169
0.15641     0.11002      0.1577    0.015011     0.12511     0.11599     0.11072    0.057124     0.12169     0.13273

• In the theorem, $X$ is a column vector of random variables while $A$ is a fixed $n\times n$ matrix of constants, that is, it is not random at all. Thus, $AX$ is a column vector whose covariance matrix is, by definition, $$\operatorname{cov}(AX) = A\operatorname{cov}(X,X^T)A^T = A \operatorname{cov}(X)A^T$$ in your notation. – Dilip Sarwate Jul 8 '14 at 14:05
• Excuse me about this simple question. Would you please verify the dimensions of both sides for me. – remo Jul 8 '14 at 14:35
• $AX$ is a $n\times 1$ matrix since it is the product of $A$, a $n\times n$ matrix, and $X$, a $n\times 1$ matrix (a.k.a. column vector). The covariance matrix of a column vector is a $n\times n$ matrix whose entries are $\operatorname{cov}(X_i,X_j)$. Thus, your $A \operatorname{cov}(X)A^T$ is the product of three $n\times n$ matrices and is thus also a $n\times n$ matrix. – Dilip Sarwate Jul 8 '14 at 14:50
• What does cov(AX) represent in this case? Is it correct to say it represents the variance of X? – Alex F Sep 1 '17 at 12:49

Let $x \in \mathbb{R}^{p \times 1}$ be a random (column) vector with covariance matrix $\Sigma \in \mathbb{R}^{p \times p}$. Let $y=Ax$, where $A \in \mathbb{R}^{n \times p}$ and $y \in \mathbb{R}^{n \times 1}$. The theorem says that $$\mathrm{Cov}(Ax) = A \Sigma A^T$$ The proof of the theorem is straightforward: $$\mathrm{Cov}(y) = \mathrm{Cov}(Ax) = \mathbb{E}[(Ax - \mathbb{E}[Ax])(Ax - \mathbb{E}[Ax])^T] \\ = \mathbb{E}[(Ax - A \mathbb{E}[x])(Ax - A\mathbb{E}[x])^T] \\ = \mathbb{E}[A (x - \mathbb{E}[x])(x - \mathbb{E}[x])^T A^T] \\ = A \underbrace{\mathbb{E}[(x - \mathbb{E}[x])(x - \mathbb{E}[x])^T]}_{=\Sigma} A^T \\$$ and this concludes the proof.
EDIT: Suppose now that $Y = AX$, where $X \in \mathbb{R}^{p \times p}$, $A \in \mathbb{R}^{n \times p}$ and $Y \in \mathbb{R}^{n \times p}$. Essentially, everything could be written in standard notation by using vectorization, i.e., defining $$x \triangleq \mathrm{vec}(X), \quad y \triangleq \mathrm{vec}(Y)$$ with $x \in \mathbb{R}^{p^2 \times 1}$, $y \in \mathbb{R}^{np \times 1}$ and writing the problem in the form $y = Hx$, where $H$ is given by $$H = \begin{bmatrix} A \\ & A \\ & & \ddots \\ & & & A \end{bmatrix} \in \mathbb{R}^{np \times p^2}$$ After that, the theorem can be applied and the proof remains practically unchanged.
• Ok, I got what you are saying. If $X$ is a matrix of random variables, you must beforehand use the "vec" operator and convert it into a column vector and adjust the model accordingly. After that, you can apply the theorem. The actual theorem is about the covariance of $Ax$ (a matrix and a column vector). If you want a formulation for $AX$, it could probably be derived, but using the "vec" operator is simpler. – PseudoRandom Jul 8 '14 at 14:45