How does the generalized linear model generalize the general linear model?

Question

From Wikipedia

The general linear model (GLM) is a statistical linear model. It may be written as1 $$ \mathbf{Y} = \mathbf{X}\mathbf{B} + \mathbf{U}, $$ where $Y$ is a matrix with series of multivariate measurements, $X$ is a matrix that might be a design matrix, $B$ is a matrix containing parameters that are usually to be estimated and $U$ is a matrix containing errors or noise. The errors are usually assumed to follow a multivariate normal distribution.

It then says

If the errors do not follow a multivariate normal distribution, generalized linear models may be used to relax assumptions about $Y$ and $U$.

I was wondering how the generalized linear models relax assumptions about $Y$ and $U$ in the general linear models?

Note that I can understand their another relation in the opposite direction:

The general linear model may be viewed as a case of the generalized linear model with identity link.

But I doubt this will help with my question.

Answer 1

Consider a case where your response variable is a set of 'successes' and 'failures' (also represented as 'yeses' and 'nos', $1$s and $0$s, etc.). If this were true, it cannot be the case that your error term is normally distributed. Instead, your error term would be Bernoulli, by definition. Thus, one of the assumptions that are alluded to is violated. Another such assumption is that of homoskedasticity, but this would be violated as well, because the variance is a function of the mean. So we can see that the (OLS) GLM is inappropriate for this case.

Note that, for a typical linear regression model, what you are predicting (i.e., $\hat y_i$) is $\mu_i$, the mean of the conditional normal distribution of the response at that exact spot where $X=x_i$. What we need in this case is to predict $\hat\pi_i$, the probability of 'success' at that spot. So we think of our response distribution as Bernoulli, and we are predicting the parameter that controls the behavior of that distribution. There is one important complication here, however. Specifically, there will be some values for $\bf X$ that, in combination with your estimates $\boldsymbol\beta$ will yield predicted values of $\hat y_i$ (i.e, $\hat\pi_i$) that will be either $<0$ or $>1$. But this is impossible, because the range of $\pi$ is $(0,~1)$. Thus we need to transform the parameter $\pi$ so that it can range $(-\infty,~\infty)$, just as the right hand side of your GLiM can. Hence, you need a link function.

At this point, we have stipulated a response distribution (Bernoulli) and a link function (perhaps the logit transformation). We already have a structural part of our model: $\bf X \boldsymbol \beta$. So now we have all the required parts of our model. This is now the generalized linear model, because we have 'relaxed' the assumptions about our response variable and the errors.

To answer your specific questions more directly, the generalized linear model relaxes assumptions about $\bf Y$ and $\bf U$ by positing a response distribution (in the exponential family) and a link function that maps the parameter in question to the interval $(-\infty,~\infty)$.

For more on this topic, it may help you to read my answer to this question: Difference between logit and probit models.