Expectation of a matrix for variance-covariance

I am implementing a variance-covariance matrix myself and I'm having some trouble understanding what is meant by the expectation this equation:

$Cov(X, X) = E[(X - E(x))(X - E(x))']$

From what I understand, the expectation of a matrix of variables is the expectation of the columns of the matrix:

$X = \left( \begin{array}{ccc} 1 & 4 & 7 \\ 2 & 5 & 8 \\ 3 & 6 & 9 \\ 4 & 3 & 2 \end{array} \right)$

$E(X) = \left(\begin{array}{ccc} 2.5 & 4.5 & 6.5 \end{array}\right)'$

I would expect the variance-covariance matrix to be a $3x3$ matrix but using this definition of expectation $(X - E(x))$ is a $4x3$ matrix, $(X - E(x))'$ is a $3x4$ matrix, $(X - E(x))(X - E(x))'$ is therefore a $3x3$ matrix but the the expectation of this is going to be a $3x1$ column vector using the above definition.

How is this outer expectation performed so the result is a $3x3$ matrix?

Expectation of a matrix of variables is not the expectation of the columns of the matrix. What may confuse you is that you treat each column as a variable and calculate it's expectation estimate like an average of it's column. In this sense you are right. However covariance matrix is about covariation between this variables. So for example cell (1,3) is covariation between the third and the first variables and so on. As you have 3 variables it can be 9 covariations (including covariation of variable with itself that is variation) that is how you get $3x3$ matrix.

So covariation matrix is:

$\begin{bmatrix}cov(x_{1},x_{1})& cov(x_{1},x_{2}) &cov(x_{1},x_{3})\\cov(x_{2},x_{1})& cov(x_{2},x_{2}) &cov(x_{2},x_{3})\\cov(x_{3},x_{1})& cov(x_{3},x_{2}) &cov(x_{3},x_{3})\end{bmatrix}$

Finally one more thing. Expectation of matrix of variables is a matrix of expectations of its cells. That is if:

$x=\begin{bmatrix}a& b\\c& d\\\end{bmatrix}$

Then

$E(x)=\begin{bmatrix}E(a)& E(b)\\E(c)& E(d)\\\end{bmatrix}$

• Thanks for your reply - since the expectation of a matrix is the expectation of it's cells, and the cells are all constant wouldn't it become the same matrix then? I apologize if I'm not following. – user124589 Dec 28 '16 at 17:43
• The cells are not the constants but also random variables. Suppose independend random variables $x\sim N(1,0)$, $y\sim Bi(10,0.3), z\sim U(0,1)$ and $t=\sim N(15,10),$ then random matrix $v=\begin{bmatrix}x& y\\z & t\end{bmatrix}$ will have expectation: $E(v)=\begin{bmatrix}1& 3\\\frac{1}{2} & 15\end{bmatrix}$ – Bogdan Dec 28 '16 at 17:51
• Also covariation is not random variable as it really constant so it's expectation equals to itself. – Bogdan Dec 28 '16 at 17:53
• thank you for the clarification. Is that why in this post: stats.seandolinar.com/making-a-covariance-matrix-in-r the author creates a "matrix of means" out of the column-wise expectation in order to properly subtract them? – user124589 Dec 28 '16 at 18:03
• Exatcly, he just gets E(x) term for covariation matrix formula you have mentioned above. – Bogdan Dec 28 '16 at 18:16