Law of Total Expectation and Law of Total Variance for Matrices Does anyone have a reference or proof for the LTE and LTV for matrices? I am defining the unconditional variance for matrices in the usual way:
$$
\operatorname{Var}_{m}(M) \overset{\text{def}}{=} \operatorname{Var}(\operatorname{vec}(M)),
$$ 
where $M$ is a random matrix, $\operatorname{vec}$ is the vectorization operator and $\operatorname{Var}$ is the variance operator for vectors. I am guessing conditional variances are defined in a similar way.
 A: I'll put the $M$ subscript to denote matrix operators.
For matrices $\mathbf{X} = [X_1\cdots X_n]$ and $\mathbf{Y} = [Y_1 \cdots Y_n]$, the law of total expectation works because we can apply the vector version (that we know to be true) to each column. 
\begin{align*}
E_M(E_M[\mathbf{X} |\mathbf{Y} ]) 
&= \left[E(E_M[\mathbf{X} \vert \mathbf{Y}]_1),\ldots, E(E_M[\mathbf{X}\vert\mathbf{Y}]_n) \right] \\
&= \left[E(E[\mathbf{X}_1\vert\mathbf{Y}]),\ldots, E(E[\mathbf{X}_n \vert\mathbf{Y}]) \right] \\
&= \left[E(\mathbf{X}_1),\ldots, E(\mathbf{X}_n) \right] \tag{vector LTE}\\
&= E[\mathbf{X}]
\end{align*}
The first two lines work because extracting columns and applying the expectation operator is the same as applying the expectation operator and then extracting columns. 
Same pattern for the LTV: I'm using the vector LTV:
\begin{align*}
&\operatorname{Var}_M\left[ E_M(\mathbf{X} \vert \mathbf{Y}) \right] + E_M\left[\operatorname{Var}_M(\mathbf{X} \vert \mathbf{Y}) \right]\\
&= \operatorname{Var}\left[ \operatorname{vec}E_M(\mathbf{X} \vert \mathbf{Y})  \right] + E_M\left[\operatorname{Var}( \operatorname{vec} \mathbf{X} \vert \mathbf{Y}) \right] \tag{defns mat var}\\
&= \operatorname{Var}\left[ E( \operatorname{vec}\mathbf{X} \vert \mathbf{Y})  \right] + E_M\left[\operatorname{Var}( \operatorname{vec} \mathbf{X} \vert \mathbf{Y}) \right] \tag{**}\\
&= \operatorname{Var}(\operatorname{vec}(\mathbf{X})) \tag{vector LTV} \\
&= \operatorname{Var}(\mathbf{X}). \tag{defn mat var}
\end{align*}
(**) works because we can interchange $\operatorname{vec}$ and $E$ (sort of, different $E$).
