# Law of Total Expectation and Law of Total Variance for Matrices

Does anyone have a reference or proof for the LTE and LTV for matrices? I am defining the unconditional variance for matrices in the usual way: $$\operatorname{Var}_{m}(M) \overset{\text{def}}{=} \operatorname{Var}(\operatorname{vec}(M)),$$ where $M$ is a random matrix, $\operatorname{vec}$ is the vectorization operator and $\operatorname{Var}$ is the variance operator for vectors. I am guessing conditional variances are defined in a similar way.

• What are you trying to ask? Matrices are vectors.
– whuber
Aug 30 '17 at 17:29
• @whuber I'm not sure if I follow. Do you mean all vectors are matrices? Aug 30 '17 at 20:19
• No: but all matrices are vectors. Automatically: you can add them and multiply them by scalars. That makes them vectors. Thus, if you have a reference for "LTE" or "LTV" or any general theorem "XYZ" whatsoever for vectors, it's automatically true for matrices.
– whuber
Aug 30 '17 at 21:11
• @whuber ok thanks. I should probably check to make sure there isn't any non-vector space stuff that creeps in anywhere Aug 30 '17 at 21:16
• Yes, that's a good idea. But in principle, if a theorem is just about vectors, it applies to all vectors in its scope. The standard pitfalls are (1) pay attention to the scalars: some theorems require them to be complex numbers, for instance, and others are limited to real numbers (algebraic closure is the operative distinction there--the ability to find roots of polynomials); (2) there's a distinction between finite-dimensional and infinite-dimensional spaces. The latter don't necessarily enjoy all the properties of the former. (They might not have a basis, for instance.)
– whuber
Aug 30 '17 at 21:19

I'll put the $M$ subscript to denote matrix operators.
For matrices $\mathbf{X} = [X_1\cdots X_n]$ and $\mathbf{Y} = [Y_1 \cdots Y_n]$, the law of total expectation works because we can apply the vector version (that we know to be true) to each column.
Same pattern for the LTV: I'm using the vector LTV: \begin{align*} &\operatorname{Var}_M\left[ E_M(\mathbf{X} \vert \mathbf{Y}) \right] + E_M\left[\operatorname{Var}_M(\mathbf{X} \vert \mathbf{Y}) \right]\\ &= \operatorname{Var}\left[ \operatorname{vec}E_M(\mathbf{X} \vert \mathbf{Y}) \right] + E_M\left[\operatorname{Var}( \operatorname{vec} \mathbf{X} \vert \mathbf{Y}) \right] \tag{defns mat var}\\ &= \operatorname{Var}\left[ E( \operatorname{vec}\mathbf{X} \vert \mathbf{Y}) \right] + E_M\left[\operatorname{Var}( \operatorname{vec} \mathbf{X} \vert \mathbf{Y}) \right] \tag{**}\\ &= \operatorname{Var}(\operatorname{vec}(\mathbf{X})) \tag{vector LTV} \\ &= \operatorname{Var}(\mathbf{X}). \tag{defn mat var} \end{align*} (**) works because we can interchange $\operatorname{vec}$ and $E$ (sort of, different $E$).