If I have a vector y which follows a multivariate normal distribution with zero mean and unknown covariance matrix. I have the following simple linear regression: $y = \beta_0 + X\beta_1 + e$.

If I consider now that the vector y follows a multivariate normal distribution with non zero mean (and unknown) and unknown covariance matrix. So what can differ in the formula of the simple linear regression??

So by assuming that the mean vector of y is unknown, what can this affect the estimation of the regression coefficients?

  • $\begingroup$ Edited my answer. $\endgroup$ – jlimahaverford Sep 13 '15 at 0:04

The normality assumption in OLS is on the error term, not the dependent variable. A non zero mean on the error term could simply be added to $\beta$ and subtracted from the error term, meaning that any model with nonzero mean is equivalent to a model with zero mean. Therefore it is reasonable to always assume this error has zero mean.

edit for detail:

Consider model 1:

$X$ and $E$ are random variables. $E$ is normal with mean $\mu$. Define the random variable

$$ Y = a + bX + E. $$

Now define $c = a + \mu$ and $D = E - \mu$. Notice that

$$ Y = c + bX + D. $$

So any relationship between $X, Y$ that can be described by a linear model with Gaussian noise, can be described by a linear model with Gaussian noise of mean zero.

  • $\begingroup$ Why you didn't also add $\mu$ to $b$? $\endgroup$ – Christina Sep 13 '15 at 0:11
  • 1
    $\begingroup$ Why would I? b is being multiplied by X. We are trying to add and subtract a constant from the RHS of the equation, to preserve equality with Y. $\endgroup$ – jlimahaverford Sep 13 '15 at 0:13
  • $\begingroup$ ah so if we assume that a is equal to zero, so we will have $Y = \mu + bX + D$. Thank you anyway :) $\endgroup$ – Christina Sep 13 '15 at 0:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.