3
$\begingroup$

This question is closely related to this question, but I do not fully understand the answer of @Taylor.

Suppose I have $N$-x-$1$ random variables $\mathbf{a}$ and $M$-x-$1$ $\mathbf{b}$ which are independent of each other with known covariances matrices $A$ and $B$ and non-zero means $E(\mathbf a)$ and $E(\mathbf b)$. What is the covariance matrix $C$ of random variable $\mathbf{c} = \mathbf{a} \otimes \mathbf{b}$ in terms of $A$ and $B$?


First, as $\mathbf a$ and $\mathbf b$ are independent, one can show that $E(c) = E(a) \otimes E(b)$. Then, the covariance matrix is \begin{align*} &E[(\mathbf{c}-E(c))E(\mathbf{c}-E(c))^T] \\ &= E\left\{ \left[ \begin{array}{c} a_1\mathbf{b} -E(a_1)E(\mathbf b)\\ a_2\mathbf{b} -E(a_2)E(\mathbf b)\\ \vdots \\ a_m\mathbf{b} -E(a_m)E(\mathbf b) \end{array}\right] \left[a_1\mathbf{b}^T - E(a_1)E(\mathbf{b})^T, \cdots, a_m\mathbf{b}^T - E(a_m)E(\mathbf{b})^T\right] \right\} \\ &= \left[\begin{array}{ccc} E[a_1a_1\mathbf{b}\mathbf{b}^T] - E(a_1)E(a_1)E(\mathbf{b})E(\mathbf{b})^T & \cdots & E[a_1a_m \mathbf{b}\mathbf{b}^T] - E(a_1)E(a_m)E(\mathbf{b})E(\mathbf{b})^T\\ \vdots & \ddots & \vdots \\ E[a_ma_1 \mathbf{b}\mathbf{b}^T] - E(a_m)E(a_1)E(\mathbf{b})E(\mathbf{b})^T & \cdots & E[a_ma_m \mathbf{b}\mathbf{b}^T] - E(a_m)E(a_m)E(\mathbf{b})E(\mathbf{b})^T \end{array} \right] \end{align*}

For the case with $E(a)=E(b)=0$ each of the expectations in the previous matrix would split up, and one would get $\operatorname{Var}(\mathbf{a}) \otimes \operatorname{Var}(\mathbf{b})$ for the covariance matrix.

But what about the case with non-centered $a$ and $b$?

$\endgroup$
  • 2
    $\begingroup$ Your notation appears to overcomplicate an inherently simple situation: you require the covariance of $a_ib_j$ and $a_kb_l$ for indexes $i,j,k,l$. The independence assumption makes it easy to compute the required moments in terms of the moments of $\mathbf a$ and $\mathbf b$. $\endgroup$ – whuber Dec 3 '17 at 16:21
3
$\begingroup$

First $$ E[\mathbf{c}] = E[\mathbf{a}\otimes \mathbf{b}] = E[\mathbf{a}] \otimes E[\mathbf{b}] = \mu_a \otimes \mu_b $$ by independence.

Then \begin{align*} \operatorname{Var}[\mathbf{c}] &= E\left[ (\mathbf{a} \otimes \mathbf{b} - \mu_a \otimes \mu_b)(\mathbf{a} \otimes \mathbf{b} - \mu_a \otimes \mu_b)'\right] \tag{defn of var}\\ &= E\left[\{(\mathbf{a}-\mu_a)\otimes (\mathbf{b} - \mu_b) \}\{(\mathbf{a}-\mu_a)\otimes (\mathbf{b} - \mu_b)\}'\right] \tag{properties of kron}\\ &= \operatorname{Var}\left[ \tilde{\mathbf{a}} \otimes \tilde{\mathbf{b}} \right] \end{align*} and we can use the linked post's result because these new vectors are zero-mean. But yeah, I think that second to last equality is worth doing out. Watch out, because you seem to have a few typos: e.g. $E[(\mathbf{c}-E(c))E(\mathbf{c}-E(c))^T]$ .

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.