2
$\begingroup$

If the dependent variable is normally distributed for a fixed set of predictor values, then the residual values should be normally distributed with a mean of 0.

I have two questions based on the above paragraph:

First, I don't understand why the above paragraph must be true. Could someone please provide more details why the above must be true? Would be great if the answer clearly states what properties of random variables are being used to arrive at the above conclusion and any other critical assumptions not explicitly stated in the paragraph. For simplicity, please assume a simple linear regression.

The second question is I am not really sure what the phrase "...for a fixed set of predictor values.." mean. Does this mean that:

Suppose we have the following pairs of observations ($x$ is the predictor value, $y$ is the dependent value):

$$( x_1 , y_1) , (x_2,y_2), (x_3,y_3)$$

Does this mean that for each $x_i$, there will be corresponding predicted value $y_i$ and all of these $y_i$ will follow a normal distribution with possibly different means for $y_i$ but same variance, where $i \in [1,3]$?

$\endgroup$
3
$\begingroup$

Assuming you are working with some model like $$ y_i = x_i'\beta + e_i$$

If you assume: $$ y_i | x_i \sim N(\mu, \sigma^2) \qquad (1)$$

This implies: $$ e_i | x_i = (y_i - x_i'\beta) | x_i$$

Since you are conditioning on $x_i$, the $x_i'\beta$ term is not random. As long as you include an intercept, $y_i - x_i'\beta$ will have mean 0. Therefore,

$$e_i | x_i \sim N(0, \sigma^2)$$

You should be aware that (1) is a very strong assumption, and the model can be estimated under different (weaker) assumptions.

The bit about fixed regressors is an assumption that I think is more common in some fields (perhaps, lab sciences) than in others (social sciences where $x$ includes covariates that are part of a sample that are random along with $y$). This assumption gets you straight-away that the distribution of $y_i|x_i$ does not depend on $x_i$. I.e. $f(y_i|x_i) = f(y_i)$ for any distribution that $y$ follows.

$\endgroup$
  • $\begingroup$ Could you please clarify why the intercept must be included so that the mean of the residuals is 0? I noticed $y_i - x_i'\beta = \epsilon_i$, so $E(y_i - x_i'\beta) = E(\epsilon_i) = 0$, since each error term having a mean of 0 is another assumption of OLS. So, is the error term, $\epsilon_i$ the intercept you are referring to? $\endgroup$ – mauna May 29 '14 at 2:51
  • 1
    $\begingroup$ The intercept is $\alpha$ in $y=\alpha+\beta x + \epsilon$, where I take a univariate regression for simplicity. Since $\alpha = \bar{y}-\beta \bar{x}$ by definition, it follows that you can write $y-\beta x-\epsilon=\alpha$. If you do not include the intercept, you get expectation of the LHS of $\alpha$; if you do include it on both sides, the expectation would be $\alpha-\alpha=0$. $\endgroup$ – Daniel Pinto May 30 '17 at 6:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.