what does the normality assumption in OLS and glm imply

Question

I am a bit confused about the normality assumption of the error term in linear regression models.

Several textbooks write that one of the Least Squares assumptions is that the (conditional) distribution of the error term is normal. Does this usually imply that the dependent variable is normally distributed itself?

This question came up while I was trying to understand generalized linear models. McCullagh and Nelder (1983, p.35) define Models for continuous data with constant variance in the following way:

As far as I understand, this should be the equivalent to the classical linear regression model, but within the framework of a glm. What I do not quite understand is the most left expression specifying that the dependent variable is normally distributed, but underneath they write "errors normally distributed and independent".

Do glms simply make stronger assumptions than would be necessary with an OLS framework? And does the dependent variable being normally distributed imply that the error terms are also normally distributed?

I would be very greatful for some enlightenment on this issue!

Answer 1

Does this usually imply that the dependent variable is normally distributed itself?

If by "dependent variable" you mean the marginal distribution, then the answer is no. the easiest counter example is a t test where the data are truly normal. The data could be bimodal and OLS/Gaussian GLM could still be applied. Here is an example of that.

What I do not quite understand is the most left expression specifying that the dependent variable is normally distributed, but underneath they write "errors normally distributed and independent".

If $\varepsilon \sim \mathcal{N}(0, \sigma)$, then $\mu + \varepsilon \sim \mathcal{N}(\mu, \sigma)$. This means that if $\mu_i = \beta_0 + \beta_1 x_i$

$$ y_i = \beta_0 + \beta_1 x_i + \varepsilon = \mu_i + \varepsilon \sim \mathcal{N}(\mu_i, \sigma)$$

The expression you've included is just another way of stating the familiar $ y_i = \beta_0 + \beta_1 x_i + \varepsilon$ assumption in OLS.

Do glms simply make stronger assumptions than would be necessary with an OLS framework?

For gaussian GLMs, no. OLS is a gaussian GLM. The test statistics you obtain are z statistics however, where as OLS uses t-statistics. We make the assumption that the z statistic is good enough as a consequence of the likelihood being asymptotically normal.

And does the dependent variable being normally distributed imply that the error terms are also normally distributed?

No. That the errors are normal is equivalent to stating the conditional distribution is normal. GLM makes no assumption on the distribution of the marginal.