Skip to main content
1 of 4
jbowman
  • 41.1k
  • 8
  • 78
  • 138

I'll assume that the elements of $\mathbf{y}$ are i.i.d. and likewise for the elements of $\mathbf{X}$. This is important, though, so be forewarned!

  1. The diagonal elements of the covariance matrix equal the sum of $m$ products of i.i.d. random variates, so the variance will equal $m \mathbb{V}(x_{ij}y_j)$, which variance you have above in your first row.

  2. The off-diagonal elements all equal zero, as the rows of $\mathbf{X}$ are independent. To see this, without loss of generality assume $\mathbb{E}x_{ij} = \mathbb{E}y_i = 0 \space \forall\thinspace i,j$. Define $\mathbf{x}_i$ as the $i^{\text{th}}$ row of $\mathbf{X}$. Then:

$\text{Cov}(\mathbf{x_iy},\mathbf{x_jy}) = \mathbb{E}(\mathbf{x_iy})^\text{T}(\mathbf{x_jy}) = \mathbb{E}\mathbf{y}^{\text{T}}\mathbf{x}_i^\text{T}\mathbf{x}_j\mathbf{y}=\mathbb{E}_y\mathbb{E}_x \mathbf{y}^{\text{T}}\mathbf{x}_i^\text{T}\mathbf{x}_j\mathbf{y}$

When $i \ne j$, the expectation with respect to $x$ of $\mathbf{y}^{\text{T}}\mathbf{x}_i^\text{T}\mathbf{x}_j\mathbf{y}$ equals 0 for any $\mathbf{y}$, as it is just the expectation of a sum of random variables each with expected value 0 (due to the assumed independence between the elements of $\mathbf{X}$). Consequently, the entire expectation equals 0.

jbowman
  • 41.1k
  • 8
  • 78
  • 138