8
$\begingroup$

From Wikipedia

The general linear model (GLM) is a statistical linear model. It may be written as1 $$ \mathbf{Y} = \mathbf{X}\mathbf{B} + \mathbf{U}, $$ where $Y$ is a matrix with series of multivariate measurements, $X$ is a matrix that might be a design matrix, $B$ is a matrix containing parameters that are usually to be estimated and $U$ is a matrix containing errors or noise. The errors are usually assumed to follow a multivariate normal distribution.

It then says

If the errors do not follow a multivariate normal distribution, generalized linear models may be used to relax assumptions about $Y$ and $U$.

I was wondering how the generalized linear models relax assumptions about $Y$ and $U$ in the general linear models?

Note that I can understand their another relation in the opposite direction:

The general linear model may be viewed as a case of the generalized linear model with identity link.

But I doubt this will help with my question.

$\endgroup$
10
$\begingroup$

Consider a case where your response variable is a set of 'successes' and 'failures' (also represented as 'yeses' and 'nos', $1$s and $0$s, etc.). If this were true, it cannot be the case that your error term is normally distributed. Instead, your error term would be Bernoulli, by definition. Thus, one of the assumptions that are alluded to is violated. Another such assumption is that of homoskedasticity, but this would be violated as well, because the variance is a function of the mean. So we can see that the (OLS) GLM is inappropriate for this case.

Note that, for a typical linear regression model, what you are predicting (i.e., $\hat y_i$) is $\mu_i$, the mean of the conditional normal distribution of the response at that exact spot where $X=x_i$. What we need in this case is to predict $\hat\pi_i$, the probability of 'success' at that spot. So we think of our response distribution as Bernoulli, and we are predicting the parameter that controls the behavior of that distribution. There is one important complication here, however. Specifically, there will be some values for $\bf X$ that, in combination with your estimates $\boldsymbol\beta$ will yield predicted values of $\hat y_i$ (i.e, $\hat\pi_i$) that will be either $<0$ or $>1$. But this is impossible, because the range of $\pi$ is $(0,~1)$. Thus we need to transform the parameter $\pi$ so that it can range $(-\infty,~\infty)$, just as the right hand side of your GLiM can. Hence, you need a link function.

At this point, we have stipulated a response distribution (Bernoulli) and a link function (perhaps the logit transformation). We already have a structural part of our model: $\bf X \boldsymbol \beta$. So now we have all the required parts of our model. This is now the generalized linear model, because we have 'relaxed' the assumptions about our response variable and the errors.

To answer your specific questions more directly, the generalized linear model relaxes assumptions about $\bf Y$ and $\bf U$ by positing a response distribution (in the exponential family) and a link function that maps the parameter in question to the interval $(-\infty,~\infty)$.

For more on this topic, it may help you to read my answer to this question: Difference between logit and probit models.

$\endgroup$
  • $\begingroup$ (+1) Concise and understandable answer. $\endgroup$ – COOLSerdash Jun 27 '13 at 12:10
  • $\begingroup$ Thanks, gung! In general linear models, "the errors are usually assumed to follow a multivariate normal distribution." Is it correct that general linear models are not necessarily parametric in the sense that they don't fully specify the form of the distribution of Y given X? Because a generalized linear model always specify the distribution of Y given X to be an exponential family distribution, is it correct that a general linear model may not be a generalized linear model? $\endgroup$ – StackExchange for All Aug 1 '13 at 13:42
  • $\begingroup$ No, the general linear model is fully specified; it is always a special case of the generalized linear model. $\endgroup$ – gung - Reinstate Monica Aug 1 '13 at 13:46
  • $\begingroup$ Does "the errors are usually assumed to follow a multivariate normal distribution" in WIkipedia for the general linear model mean that the error may not be normally distributed? $\endgroup$ – StackExchange for All Aug 1 '13 at 13:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.