1
$\begingroup$

I have a doubt about the distribution of the error in the standard linear regression framework.

$ Y = \beta X + e$

What is the hypotesis about? the error? the dependent variable Y? the regressors X?

if I say that the error is normally distributed since $e = Y-\beta X$ I am implying that:

$Y-\beta X$ is normal distributed so (Y,X) are jointly normal and the other way around.

if I say that $Y$ is normally distributed then I am implying that $\beta X + e$ is normal and so (X,e) are jointly normal.

So making the assumption of normality about Y or X ore e is the same thing. Am I right?

$\endgroup$
4
$\begingroup$

The assumption is specifically about errors $e$. This has nothing to do with $X$ and $Y$. You simply assume that $e$ are random; that's important. Errors $e$ may be normal, maybe not, it's not that important, usually.

There's an assumption about $X$ and $e$ being uncorrelated or independent, which is important.

In any case these don't follow from anything. You simply assume these are true. Whether your assumption is grounded in truth or not is a different story. For instance, often errors $e$ are correlated with $X$ in practice.

$\endgroup$
  • $\begingroup$ I the econometric book of Bruce Hansen (Econometrics) chapter 2 page 38 par 2.25, he says that the assumption is about Y and as I wrote the fact that the error is normal is simply a consequence of Y assumed to be normal $\endgroup$ – Hard Core Sep 18 '17 at 15:37
  • $\begingroup$ @HardCore, consider $Y_t=t+e, e\sim\mathcal{N}(0,1)$ would you call this Y normal? $\endgroup$ – Aksakal Sep 18 '17 at 15:39
  • $\begingroup$ Of course Y is a normal where the mean is shifted of t. A normal plus a a quantity t is again normal $\endgroup$ – Hard Core Sep 18 '17 at 15:46
  • $\begingroup$ @HardCore, you mus qualify this by saying it's conditionally normal, conditioning on $t$. It's not normal unconditionally. $\endgroup$ – Aksakal Sep 18 '17 at 15:48
  • 1
    $\begingroup$ @HardCore, from your textbook section it looks like the author starts by assuming the variables (y, x) are jointly normally distributed and uses that to show that e ∼ N(0, σ^2). Which implies that y is conditionally normal. But if x ∼ N(0, σ^2), that does not imply that e ∼ N(0, σ^2) since e is independent of x. This might help: ics.uci.edu/~jutts/110-201-08/Oct13Lecture.pdf $\endgroup$ – Great38 Sep 18 '17 at 17:35
1
$\begingroup$

In a regression model, the covariates ($x$s) are not random variables, they are fixed values on which we are conditioning the expectation of $Y$.

$$E(Y | x_1, x_2, ..., x_n)$$

So,

$E \sim \text{Normal}$

because

$Y \sim \text{Normal}$

and

$E = Y-\beta X$

which can be broken down like this:

$E = \text{random component} + \text{non-random component}$

$\endgroup$
  • 1
    $\begingroup$ "the covariates (xxs) are not random variables" - not entirely true. It is *usually" assumed that X is fixed, e.g. in textbooks, but it's not necessary and in some applications they are random, so you have to adjust the assumptions $\endgroup$ – Aksakal Sep 18 '17 at 14:53
  • 2
    $\begingroup$ This is very sloppy: $E\sim Normal$ because $Y\sim Normal$. Y is most certainly not normal in a usual sense unless you have intercept only model. $\endgroup$ – Aksakal Sep 18 '17 at 14:56
  • 1
    $\begingroup$ Notation could be clearer here. You're equivocating between $E$ as expectation operator and the result of taking that expectation. It's a case of people who understand this know what you mean, but is it clear to others? $\endgroup$ – Nick Cox Sep 18 '17 at 15:09
  • 1
    $\begingroup$ @HardCore, covariates X are often fixed, that's the term. For instance, in natural sciences and engineering you control the experiment, you set the temperature -> you set X fixed. In social sciences you usually do not control the conditions, they're what you observe, e.g. you measure GDP as a covariate, but you can't set it to a level that you choose. It is what it is, hence, the covariates X are random in this case. $\endgroup$ – Aksakal Sep 18 '17 at 15:19
  • 2
    $\begingroup$ @HardCore, in my past life as a physicist my regressors were always fixed. I didn't do any observational studies. Sometimes, in marketing applications the regressors can be fixed, if you're able to conduct experiments $\endgroup$ – Aksakal Sep 18 '17 at 15:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.