1
$\begingroup$

I am a bit confused about the normality assumption of the error term in linear regression models.

Several textbooks write that one of the Least Squares assumptions is that the (conditional) distribution of the error term is normal. Does this usually imply that the dependent variable is normally distributed itself?

This question came up while I was trying to understand generalized linear models. McCullagh and Nelder (1983, p.35) define Models for continuous data with constant variance in the following way:

enter image description here

As far as I understand, this should be the equivalent to the classical linear regression model, but within the framework of a glm. What I do not quite understand is the most left expression specifying that the dependent variable is normally distributed, but underneath they write "errors normally distributed and independent".

Do glms simply make stronger assumptions than would be necessary with an OLS framework? And does the dependent variable being normally distributed imply that the error terms are also normally distributed?

I would be very greatful for some enlightenment on this issue!

$\endgroup$
4
$\begingroup$

Does this usually imply that the dependent variable is normally distributed itself?

If by "dependent variable" you mean the marginal distribution, then the answer is no. the easiest counter example is a t test where the data are truly normal. The data could be bimodal and OLS/Gaussian GLM could still be applied. Here is an example of that.

What I do not quite understand is the most left expression specifying that the dependent variable is normally distributed, but underneath they write "errors normally distributed and independent".

If $\varepsilon \sim \mathcal{N}(0, \sigma)$, then $\mu + \varepsilon \sim \mathcal{N}(\mu, \sigma)$. This means that if $\mu_i = \beta_0 + \beta_1 x_i$

$$ y_i = \beta_0 + \beta_1 x_i + \varepsilon = \mu_i + \varepsilon \sim \mathcal{N}(\mu_i, \sigma)$$

The expression you've included is just another way of stating the familiar $ y_i = \beta_0 + \beta_1 x_i + \varepsilon$ assumption in OLS.

Do glms simply make stronger assumptions than would be necessary with an OLS framework?

For gaussian GLMs, no. OLS is a gaussian GLM. The test statistics you obtain are z statistics however, where as OLS uses t-statistics. We make the assumption that the z statistic is good enough as a consequence of the likelihood being asymptotically normal.

And does the dependent variable being normally distributed imply that the error terms are also normally distributed?

No. That the errors are normal is equivalent to stating the conditional distribution is normal. GLM makes no assumption on the distribution of the marginal.

$\endgroup$
7
  • $\begingroup$ No. We don't usually concern ourselves with the distribution of the covariates since we treat them as known. - True, but the question is about the distribution of the dependent variables, which we do assume to be Normally distributed conditional on x. $\endgroup$
    – Firebug
    Sep 30 '20 at 14:03
  • $\begingroup$ And, on the last part, having the dependent variable be Normal conditional on x does imply the error terms are also normal with mean 0. $\endgroup$
    – Firebug
    Sep 30 '20 at 14:05
  • $\begingroup$ @Firebug if known covariates are assumed dependent variable is also unconditionally normal, because there is actually nothing to condition on. For each data point there is a different distribution. $\endgroup$ Sep 30 '20 at 14:13
  • $\begingroup$ @Firebug, oops, must have misread that. let me edit. Still early where I am, haven't finished my coffee. $\endgroup$ Sep 30 '20 at 14:13
  • $\begingroup$ @DemetriPananos it's fine, good answer +1 :) $\endgroup$
    – Firebug
    Sep 30 '20 at 14:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.