Variance of linear combinations of Normal RVs in linear regression

Question

I am working with a standard multivariate linear regression model ($Y = X \lambda + \epsilon$, $Y$ and $\epsilon$ of length $n$, $X$ is $n$ x $m$, $\lambda$ of length $m$). $X$ is said to be "column-standardized", which I take to mean each column has mean $0$ and variance $1$, and each $\epsilon_i \sim N(0, \sigma^2)$. I am looking for the distribution of $X^T \epsilon$. Specifically, I have two questions.

1. If $X$ is column-standardized, does that mean each $X_{ij} \sim N(0, 1)$? It seems like it should, since this is just a z-transformation, but then I'm curious as to why the procedure needs to be done on columns separately. Is it because we assume that each column comes from a unique distribution with different original variances?

2. If every column in standardized, then each entry $(X^T\epsilon)_j = \sum_{i=1}^{n} X_{ij} \epsilon_i$, which I think is a linear combination of standard normal distributions where the coefficients are also normal. Sums of normals are normal, and so I think that $(X^T\epsilon)_j \sim N(0, \_)$, but I'm having a hard time figuring out the variance. I don't think it's $n\sigma^2$, but I can't figure out what else it would be.

Any help would be greatly appreciated!

Answer 1

If the $X$'s are assumed independent from the $\epsilon$'s, then simple algebra produces a nice looking formula, irrespective of distributional assumptions. In general notation,

$$\text{Var}(XY) = E(X^2Y^2) - [E(XY)]^2 = E(X^2)E(Y^2) - [E(X)E(Y)]^2$$

Using the symbols $\sigma^2$ and $\mu$ we have

$$\text{Var}(XY) = (\sigma^2_x +\mu_x^2)(\sigma^2_y+\mu_y^2) - \mu_x^2\mu_y^2$$

and simplifying and rearranging,

$$\text{Var}(XY) = \sigma^2_x\sigma^2_y + \mu_x^2\sigma^2_y+\mu_y^2\sigma^2_x$$

In case of zero means, this reduces to

$$\text{Var}(XY) = \sigma^2_x\sigma^2_y $$

Let's not forget that the fact that we obtained $\text{Var}(XY) = \text{Var}(X)\text{Var}(Y) $ depends not only on the independence assumption, but also on the zero-mean assumption.