I am working with a standard multivariate linear regression model ($Y = X \lambda + \epsilon$, $Y$ and $\epsilon$ of length $n$, $X$ is $n$ x $m$, $\lambda$ of length $m$). $X$ is said to be "column-standardized", which I take to mean each column has mean $0$ and variance $1$, and each $\epsilon_i \sim N(0, \sigma^2)$. I am looking for the distribution of $X^T \epsilon$. Specifically, I have two questions.

  1. If $X$ is column-standardized, does that mean each $X_{ij} \sim N(0, 1)$? It seems like it should, since this is just a z-transformation, but then I'm curious as to why the procedure needs to be done on columns separately. Is it because we assume that each column comes from a unique distribution with different original variances?

  2. If every column in standardized, then each entry $(X^T\epsilon)_j = \sum_{i=1}^{n} X_{ij} \epsilon_i$, which I think is a linear combination of standard normal distributions where the coefficients are also normal. Sums of normals are normal, and so I think that $(X^T\epsilon)_j \sim N(0, \_)$, but I'm having a hard time figuring out the variance. I don't think it's $n\sigma^2$, but I can't figure out what else it would be.

Any help would be greatly appreciated!

  • $\begingroup$ You have a product of two independent normal random variables in $X_ij\epsilon_i$, See e.g. here for the formula for the variance of the product. Then, use standard formulas for the variance of a sum of independent random variables. $\endgroup$ – Dilip Sarwate Jun 30 '18 at 15:55
  • 1
    $\begingroup$ Standardizing a variable does not make it normally distributed. $\endgroup$ – Alecos Papadopoulos Jun 30 '18 at 17:44

If the $X$'s are assumed independent from the $\epsilon$'s, then simple algebra produces a nice looking formula, irrespective of distributional assumptions. In general notation,

$$\text{Var}(XY) = E(X^2Y^2) - [E(XY)]^2 = E(X^2)E(Y^2) - [E(X)E(Y)]^2$$

Using the symbols $\sigma^2$ and $\mu$ we have

$$\text{Var}(XY) = (\sigma^2_x +\mu_x^2)(\sigma^2_y+\mu_y^2) - \mu_x^2\mu_y^2$$

and simplifying and rearranging,

$$\text{Var}(XY) = \sigma^2_x\sigma^2_y + \mu_x^2\sigma^2_y+\mu_y^2\sigma^2_x$$

In case of zero means, this reduces to

$$\text{Var}(XY) = \sigma^2_x\sigma^2_y $$

Let's not forget that the fact that we obtained $\text{Var}(XY) = \text{Var}(X)\text{Var}(Y) $ depends not only on the independence assumption, but also on the zero-mean assumption.

  • $\begingroup$ This is essentially what Macro wrote in the link provided in my comment on the OP's question. $\endgroup$ – Dilip Sarwate Jun 30 '18 at 21:05
  • $\begingroup$ Thanks very much Alecos and Dilip. Following up -- so $E(X_{ij}\epsilon_i)=0$ and $Var(X_{ij}\epsilon_i)=\sigma^2$, but $X_{ij}\epsilon_i$ isn't necessarily the product of two normals. Do you have a sense of what kind of distribution it might follow? $\endgroup$ – Benjamin Doughty Jul 2 '18 at 4:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.