Does anyone have a reference or proof for the LTE and LTV for matrices? I am defining the unconditional variance for matrices in the usual way: $$ \operatorname{Var}_{m}(M) \overset{\text{def}}{=} \operatorname{Var}(\operatorname{vec}(M)), $$ where $M$ is a random matrix, $\operatorname{vec}$ is the vectorization operator and $\operatorname{Var}$ is the variance operator for vectors. I am guessing conditional variances are defined in a similar way.

  • $\begingroup$ What are you trying to ask? Matrices are vectors. $\endgroup$
    – whuber
    Aug 30 '17 at 17:29
  • $\begingroup$ @whuber I'm not sure if I follow. Do you mean all vectors are matrices? $\endgroup$
    – Taylor
    Aug 30 '17 at 20:19
  • $\begingroup$ No: but all matrices are vectors. Automatically: you can add them and multiply them by scalars. That makes them vectors. Thus, if you have a reference for "LTE" or "LTV" or any general theorem "XYZ" whatsoever for vectors, it's automatically true for matrices. $\endgroup$
    – whuber
    Aug 30 '17 at 21:11
  • $\begingroup$ @whuber ok thanks. I should probably check to make sure there isn't any non-vector space stuff that creeps in anywhere $\endgroup$
    – Taylor
    Aug 30 '17 at 21:16
  • 1
    $\begingroup$ Yes, that's a good idea. But in principle, if a theorem is just about vectors, it applies to all vectors in its scope. The standard pitfalls are (1) pay attention to the scalars: some theorems require them to be complex numbers, for instance, and others are limited to real numbers (algebraic closure is the operative distinction there--the ability to find roots of polynomials); (2) there's a distinction between finite-dimensional and infinite-dimensional spaces. The latter don't necessarily enjoy all the properties of the former. (They might not have a basis, for instance.) $\endgroup$
    – whuber
    Aug 30 '17 at 21:19

I'll put the $M$ subscript to denote matrix operators.

For matrices $\mathbf{X} = [X_1\cdots X_n]$ and $\mathbf{Y} = [Y_1 \cdots Y_n]$, the law of total expectation works because we can apply the vector version (that we know to be true) to each column.

\begin{align*} E_M(E_M[\mathbf{X} |\mathbf{Y} ]) &= \left[E(E_M[\mathbf{X} \vert \mathbf{Y}]_1),\ldots, E(E_M[\mathbf{X}\vert\mathbf{Y}]_n) \right] \\ &= \left[E(E[\mathbf{X}_1\vert\mathbf{Y}]),\ldots, E(E[\mathbf{X}_n \vert\mathbf{Y}]) \right] \\ &= \left[E(\mathbf{X}_1),\ldots, E(\mathbf{X}_n) \right] \tag{vector LTE}\\ &= E[\mathbf{X}] \end{align*} The first two lines work because extracting columns and applying the expectation operator is the same as applying the expectation operator and then extracting columns.

Same pattern for the LTV: I'm using the vector LTV: \begin{align*} &\operatorname{Var}_M\left[ E_M(\mathbf{X} \vert \mathbf{Y}) \right] + E_M\left[\operatorname{Var}_M(\mathbf{X} \vert \mathbf{Y}) \right]\\ &= \operatorname{Var}\left[ \operatorname{vec}E_M(\mathbf{X} \vert \mathbf{Y}) \right] + E_M\left[\operatorname{Var}( \operatorname{vec} \mathbf{X} \vert \mathbf{Y}) \right] \tag{defns mat var}\\ &= \operatorname{Var}\left[ E( \operatorname{vec}\mathbf{X} \vert \mathbf{Y}) \right] + E_M\left[\operatorname{Var}( \operatorname{vec} \mathbf{X} \vert \mathbf{Y}) \right] \tag{**}\\ &= \operatorname{Var}(\operatorname{vec}(\mathbf{X})) \tag{vector LTV} \\ &= \operatorname{Var}(\mathbf{X}). \tag{defn mat var} \end{align*} (**) works because we can interchange $\operatorname{vec}$ and $E$ (sort of, different $E$).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.