# Covariance of $a \otimes b$ for independent random variables $a$ and $b$

This question is closely related to this question, but I do not fully understand the answer of @Taylor.

Suppose I have $N$-x-$1$ random variables $\mathbf{a}$ and $M$-x-$1$ $\mathbf{b}$ which are independent of each other with known covariances matrices $A$ and $B$ and non-zero means $E(\mathbf a)$ and $E(\mathbf b)$. What is the covariance matrix $C$ of random variable $\mathbf{c} = \mathbf{a} \otimes \mathbf{b}$ in terms of $A$ and $B$?

First, as $\mathbf a$ and $\mathbf b$ are independent, one can show that $E(c) = E(a) \otimes E(b)$. Then, the covariance matrix is \begin{align*} &E[(\mathbf{c}-E(c))E(\mathbf{c}-E(c))^T] \\ &= E\left\{ \left[ \begin{array}{c} a_1\mathbf{b} -E(a_1)E(\mathbf b)\\ a_2\mathbf{b} -E(a_2)E(\mathbf b)\\ \vdots \\ a_m\mathbf{b} -E(a_m)E(\mathbf b) \end{array}\right] \left[a_1\mathbf{b}^T - E(a_1)E(\mathbf{b})^T, \cdots, a_m\mathbf{b}^T - E(a_m)E(\mathbf{b})^T\right] \right\} \\ &= \left[\begin{array}{ccc} E[a_1a_1\mathbf{b}\mathbf{b}^T] - E(a_1)E(a_1)E(\mathbf{b})E(\mathbf{b})^T & \cdots & E[a_1a_m \mathbf{b}\mathbf{b}^T] - E(a_1)E(a_m)E(\mathbf{b})E(\mathbf{b})^T\\ \vdots & \ddots & \vdots \\ E[a_ma_1 \mathbf{b}\mathbf{b}^T] - E(a_m)E(a_1)E(\mathbf{b})E(\mathbf{b})^T & \cdots & E[a_ma_m \mathbf{b}\mathbf{b}^T] - E(a_m)E(a_m)E(\mathbf{b})E(\mathbf{b})^T \end{array} \right] \end{align*}

For the case with $E(a)=E(b)=0$ each of the expectations in the previous matrix would split up, and one would get $\operatorname{Var}(\mathbf{a}) \otimes \operatorname{Var}(\mathbf{b})$ for the covariance matrix.

But what about the case with non-centered $a$ and $b$?

• Your notation appears to overcomplicate an inherently simple situation: you require the covariance of $a_ib_j$ and $a_kb_l$ for indexes $i,j,k,l$. The independence assumption makes it easy to compute the required moments in terms of the moments of $\mathbf a$ and $\mathbf b$. – whuber Dec 3 '17 at 16:21

First $$E[\mathbf{c}] = E[\mathbf{a}\otimes \mathbf{b}] = E[\mathbf{a}] \otimes E[\mathbf{b}] = \mu_a \otimes \mu_b$$ by independence.
Then \begin{align*} \operatorname{Var}[\mathbf{c}] &= E\left[ (\mathbf{a} \otimes \mathbf{b} - \mu_a \otimes \mu_b)(\mathbf{a} \otimes \mathbf{b} - \mu_a \otimes \mu_b)'\right] \tag{defn of var}\\ &= E\left[\{(\mathbf{a}-\mu_a)\otimes (\mathbf{b} - \mu_b) \}\{(\mathbf{a}-\mu_a)\otimes (\mathbf{b} - \mu_b)\}'\right] \tag{properties of kron}\\ &= \operatorname{Var}\left[ \tilde{\mathbf{a}} \otimes \tilde{\mathbf{b}} \right] \end{align*} and we can use the linked post's result because these new vectors are zero-mean. But yeah, I think that second to last equality is worth doing out. Watch out, because you seem to have a few typos: e.g. $E[(\mathbf{c}-E(c))E(\mathbf{c}-E(c))^T]$ .