I was trying to understand the difference between a multiple linear regression model, a general linear model, and a generalized linear model. I have seen very similar questions have already been answered, however I can't understand a thing. From what I see on Wikipedia:

The general linear model or general multivariate regression model is a compact way of simultaneously writing several multiple linear regression models. In that sense it is not a separate statistical linear model. The various multiple linear regression models may be compactly written as $$\mathbf {Y} =\mathbf {X} \mathbf {B} +\mathbf {U}$$

So it seem to me that a general linear model is just the same as a system of models: $$ \left\{ \begin{array}{c} y_1 = b_{11}x_1+b_{12}x_2+ \dots +b_{1n}x_n+\varepsilon_1 \\ y_2 = b_{21}x_1+b_{22}x_2+ \dots +b_{2n}x_n+\varepsilon_2 \\ \vdots\\ y_m = b_{m1}x_m+b_{m2}x_2+ \dots +b_{3n}x_n+\varepsilon_m \end{array} \right. $$ where all the $\varepsilon_i$ error terms have normal distributions.

On the other hand, a generalized linear model has no definition on the number of outputs (may have one or multiple), nor on the number of inputs, but the error variables may have distributions from all the exponential family (and that is the advantage over normal regression).

Is this the difference?

Finally, a multiple linear regression model is somewhat the dual of a general model: it has only one output, but multiple inputs (and still normal error).

Is this all correct?


1 Answer 1


I think the way that's written on Wikipedia is a little bit confusing, because it shifts between 'general linear model' and 'multivariate regression model', which are (potentially somewhat) different animals.

It is unfortunate, but the terminology in statistics is often inconsistent and simply inherited from the time when people were first working things out. Here are a couple of points that may help you to understand. Be aware that they adhere strictly to old usages, and not everyone still uses the terms these ways.

  • An ANOVA has only categorical explanatory variables and a single continuous response variable.
  • A simple regression model has a singe continuous explanatory variable and a single continuous response variable.
  • A multiple regression model has multiple continuous explanatory variables and a single continuous response variable.
  • A regression model with a single continuous response variable, but with both categorical and continuous explanatory variables is an ANCOVA, if there are no interaction terms, or a general linear model, if there are interactions. (Nowadays, most people would say that everything mentioned above is one or another special case of the general linear model.)
  • A multivariate model, say multivariate linear regression, is like the above, but it has multiple response ($Y$) variables.
  • A generalized linear model, um, generalizes the general linear model, in that the response variable need not be conditionally normal. Instead, $Y$ can be distributed as any distribution that falls within the exponential family of distributions. For example, the response distribution could be binomial (yes/no) or Poisson (counts). That said, you could have either categorical explanatory variables, continuous ones, or both (and interactions are fine, too). Since the normal distribution falls within the exponential family, all of the above are considered special cases of the generalized linear model.
  • $\begingroup$ Thanks! One thing: when you say "the response variable need not be conditionally normal", you mean that the error of the response variable (i.e. the residuals in a linear model) are normally distributed around the predicted value? Or, in other words, that the response variable follows a normal distribution, whose mean is the predicted outcome? $\endgroup$ Commented Sep 14, 2022 at 10:04
  • 1
    $\begingroup$ @AlessandroBertulli, yes. The errors (residuals) are assumed to be normally distributed w/ a mean equal to the predicted value & w/ constant variance in a linear model. $\endgroup$ Commented Sep 14, 2022 at 11:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.