If we assume a vector $y$ which has normal distribution with mean m and covariance matrix M.

A simple linear regression model is written as $y = \beta_0 + X\beta + e$.

If the mean of y is non zero and has to be estimated, so what can be its estimated value?

In other terms, is there a relationship between the mean vector and the estimated regression coefficients?

Furthermore, what is the $\beta_0$ ? why they consider in most cases that it is equal to 0?

Any help will be very appreciated!


Let me address your questions in inverse order.

Firstly, in most sources I have encountered so far the task is more conveniently reformulated as $y = X^*\beta^* + \epsilon$, where $X^*$ is just $X$ with a column vector of ones appended to the right and $\beta^*$ is your $\beta$ with $\beta_0$ appended to it. In such way, you essentially consider the same model but the explicit intercept term $\beta_0$ now has disappeared. Maybe that's what you took as $\beta_0=0$?

Secondly, the most straightforward and classic way to estimate the explanatory variables $\beta$ is to use the Least Squares method. The solution is then given in terms of the right Moore - Penrose pseudoinverse:

$$ \beta^*_{LS}=(X^{*T}X^*)^{-1}X^{*T}y $$

As you can see, the sample mean is not explicitly present here.

However, I may be missing something obvious in your question so please correct me if I'm wrong.

EDIT: After reading your question again I've noticed that in your original formulation the errors $e$ are not uncorrelated but rather have a covariance matrix $M$. So, to stay formal I would like to notice that this case can actually be reduced to the non-corellated case by a linear change of variables: write your errors as $e=\sqrt{M}e^*$, where $\sqrt{M}$ is, for example, the Cholesky square root of $M$ and $e^* \sim N(0, I)$, and multiply by $(\sqrt{M})^{-1}$.

| cite | improve this answer | |
  • $\begingroup$ Thank you for the answer. But what is the estimated value of the mean? does is just the sample mean computed as the maximum likelihood under gaussian hypothesis? $\endgroup$ – Christina Sep 4 '15 at 22:23
  • $\begingroup$ @Christina Taking the expectation of your equation, we immediately find that the expectation of y is (X*)*(\beta*). In order to estimate it, you can simply calculate the sample mean which is asymptotically the best possible unbiased estimator. However, by doing so you would not solve the regression task because you would only estimate (X*)*(\beta*) and not \beta* itself. $\endgroup$ – Vossler Sep 4 '15 at 22:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.