# Why cov(AX)=A cov(X) A'

I cannot verify the following theorem. Maybe I am doing something wrong, but I don't know what?! Additionally, I'm not sure about the meaning of a constant matrix in the theorem.

Theorem: cov(AX)=Acov(X)A' if A is a constant matrix. (A' means transpose of A, i.e., A^T)

Example:

A (10x5):

0.8727    0.7145    0.9147    0.6370    0.9760
0.8241    0.6210    0.0735    0.7318    0.1231
0.8609    0.7410    0.9865    0.6090    0.7693
0.1135    0.9563    0.2369    0.5369    0.9876
0.7349    0.4313    0.9650    0.1657    0.4418
0.7872    0.6589    0.4490    0.5974    0.7669
0.4247    0.6411    0.9335    0.5238    0.9153
0.2126    0.6212    0.6423    0.3651    0.9752
0.5092    0.6179    0.3732    0.9569    0.1673
0.8009    0.4691    0.4420    0.6289    0.3132

X (5x5):

0.6635    0.6323    0.5883    0.9362    0.0998
0.8298    0.3788    0.4760    0.8170    0.3296
0.8553    0.6836    0.4701    0.5723    0.6803
0.0065    0.8085    0.1542    0.5780    0.7653
0.6389    0.9297    0.1418    0.0438    0.0886

AX (10x5):

2.5820    2.8701    1.5202    2.3352    1.5188
1.2083    1.5127    0.9452    1.7493    0.9079
2.5253    2.7069    1.5259    2.3616    1.5355
1.7059    1.9482    0.8562    1.3768    0.9861
1.9542    1.8325    1.1795    1.7078    1.0380
1.9469    2.2502    1.1887    1.9111    1.1263
2.2004    2.4239    1.2045    1.7985    1.3707
1.8312    2.0105    0.9173    1.3278    1.0287
1.2828    1.7403    0.9404    1.7555    1.2554
1.5029    1.7859    1.0436    1.7633    1.0443

cov(X) 5x5:

0.1190   -0.0345    0.0457    0.0139   -0.0482
-0.0345    0.0429   -0.0316   -0.0557    0.0015
0.0457   -0.0316    0.0419    0.0563   -0.0132
0.0139   -0.0557    0.0563    0.1175    0.0102
-0.0482    0.0015   -0.0132    0.0102    0.1009

cov(AX) (5x5):

0.2227    0.1984    0.0988    0.1011    0.0827
0.1984    0.1960    0.0916    0.1063    0.0874
0.0988    0.0916    0.0576    0.0745    0.0455
0.1011    0.1063    0.0745    0.1144    0.0618
0.0827    0.0874    0.0455    0.0618    0.0510

but A*cov(X)*A' (10x10):

0.2195     0.12119     0.20872    0.077985     0.15076     0.16413     0.17298     0.11964     0.14558     0.15641
0.12119     0.09362     0.12556  -0.0017266     0.10372     0.08966    0.081364    0.034694    0.099912     0.11002
0.20872     0.12556     0.20256    0.057506     0.15142     0.15574     0.15915     0.10193     0.14583      0.1577
0.077985  -0.0017266    0.057506     0.11155    0.015719    0.059279    0.089875    0.099947    0.026385    0.015011
0.15076     0.10372     0.15142    0.015719     0.12378     0.11292     0.10434    0.055612     0.10669     0.12511
0.16413     0.08966     0.15574    0.059279     0.11292     0.12295     0.12914    0.090107     0.10648     0.11599
0.17298    0.081364     0.15915    0.089875     0.10434     0.12914     0.14732     0.11382     0.11134     0.11072
0.11964    0.034694     0.10193    0.099947    0.055612    0.090107     0.11382     0.10483    0.061491    0.057124
0.14558    0.099912     0.14583    0.026385     0.10669     0.10648     0.11134    0.061491     0.12616     0.12169
0.15641     0.11002      0.1577    0.015011     0.12511     0.11599     0.11072    0.057124     0.12169     0.13273

• In the theorem, $X$ is a column vector of random variables while $A$ is a fixed $n\times n$ matrix of constants, that is, it is not random at all. Thus, $AX$ is a column vector whose covariance matrix is, by definition, $$\operatorname{cov}(AX) = A\operatorname{cov}(X,X^T)A^T = A \operatorname{cov}(X)A^T$$ in your notation. Commented Jul 8, 2014 at 14:05
– remo
Commented Jul 8, 2014 at 14:35
• $AX$ is a $n\times 1$ matrix since it is the product of $A$, a $n\times n$ matrix, and $X$, a $n\times 1$ matrix (a.k.a. column vector). The covariance matrix of a column vector is a $n\times n$ matrix whose entries are $\operatorname{cov}(X_i,X_j)$. Thus, your $A \operatorname{cov}(X)A^T$ is the product of three $n\times n$ matrices and is thus also a $n\times n$ matrix. Commented Jul 8, 2014 at 14:50
• What does cov(AX) represent in this case? Is it correct to say it represents the variance of X? Commented Sep 1, 2017 at 12:49

Let $x \in \mathbb{R}^{p \times 1}$ be a random (column) vector with covariance matrix $\Sigma \in \mathbb{R}^{p \times p}$. Let $y=Ax$, where $A \in \mathbb{R}^{n \times p}$ and $y \in \mathbb{R}^{n \times 1}$. The theorem says that $$\mathrm{Cov}(Ax) = A \Sigma A^T$$ The proof of the theorem is straightforward: $$\mathrm{Cov}(y) = \mathrm{Cov}(Ax) = \mathbb{E}[(Ax - \mathbb{E}[Ax])(Ax - \mathbb{E}[Ax])^T] \\ = \mathbb{E}[(Ax - A \mathbb{E}[x])(Ax - A\mathbb{E}[x])^T] \\ = \mathbb{E}[A (x - \mathbb{E}[x])(x - \mathbb{E}[x])^T A^T] \\ = A \underbrace{\mathbb{E}[(x - \mathbb{E}[x])(x - \mathbb{E}[x])^T]}_{=\Sigma} A^T \\$$ and this concludes the proof.

EDIT: Suppose now that $Y = AX$, where $X \in \mathbb{R}^{p \times p}$, $A \in \mathbb{R}^{n \times p}$ and $Y \in \mathbb{R}^{n \times p}$. Essentially, everything could be written in standard notation by using vectorization, i.e., defining $$x \triangleq \mathrm{vec}(X), \quad y \triangleq \mathrm{vec}(Y)$$ with $x \in \mathbb{R}^{p^2 \times 1}$, $y \in \mathbb{R}^{np \times 1}$ and writing the problem in the form $y = Hx$, where $H$ is given by $$H = \begin{bmatrix} A \\ & A \\ & & \ddots \\ & & & A \end{bmatrix} \in \mathbb{R}^{np \times p^2}$$ After that, the theorem can be applied and the proof remains practically unchanged.

• Why X must be symmetric. I know cov(X) must be symmetric, but what about X, itself?
– remo
Commented Jul 8, 2014 at 14:33
• In your code, you were using "X" as a covariance matrix (look at the first comment). What I meant was: the covariance matrix must be symmetric. Commented Jul 8, 2014 at 14:35
• I cannot find such a comment, By the way I added cov(X) to my question and I mean X is a matrix of variables not necessarily symmetric. If there is such a formulation for cov(AX)? (Please see X as a matrix and not a random variable)
– remo
Commented Jul 8, 2014 at 14:42
• Ok, I got what you are saying. If $X$ is a matrix of random variables, you must beforehand use the "vec" operator and convert it into a column vector and adjust the model accordingly. After that, you can apply the theorem. The actual theorem is about the covariance of $Ax$ (a matrix and a column vector). If you want a formulation for $AX$, it could probably be derived, but using the "vec" operator is simpler. Commented Jul 8, 2014 at 14:45
• I have substantially edited my answer. Tell me if it answers your question or if there is something more you would like to see. Commented Jul 8, 2014 at 15:49