Is a general linear model a "system" of linear regression models?

Question

I was trying to understand the difference between a multiple linear regression model, a general linear model, and a generalized linear model. I have seen very similar questions have already been answered, however I can't understand a thing. From what I see on Wikipedia:

The general linear model or general multivariate regression model is a compact way of simultaneously writing several multiple linear regression models. In that sense it is not a separate statistical linear model. The various multiple linear regression models may be compactly written as $$\mathbf {Y} =\mathbf {X} \mathbf {B} +\mathbf {U}$$

So it seem to me that a general linear model is just the same as a system of models: $$ \left\{ \begin{array}{c} y_1 = b_{11}x_1+b_{12}x_2+ \dots +b_{1n}x_n+\varepsilon_1 \\ y_2 = b_{21}x_1+b_{22}x_2+ \dots +b_{2n}x_n+\varepsilon_2 \\ \vdots\\ y_m = b_{m1}x_m+b_{m2}x_2+ \dots +b_{3n}x_n+\varepsilon_m \end{array} \right. $$ where all the $\varepsilon_i$ error terms have normal distributions.

On the other hand, a generalized linear model has no definition on the number of outputs (may have one or multiple), nor on the number of inputs, but the error variables may have distributions from all the exponential family (and that is the advantage over normal regression).

Is this the difference?

Finally, a multiple linear regression model is somewhat the dual of a general model: it has only one output, but multiple inputs (and still normal error).

Is this all correct?

Answer 1

I think the way that's written on Wikipedia is a little bit confusing, because it shifts between 'general linear model' and 'multivariate regression model', which are (potentially somewhat) different animals.

It is unfortunate, but the terminology in statistics is often inconsistent and simply inherited from the time when people were first working things out. Here are a couple of points that may help you to understand. Be aware that they adhere strictly to old usages, and not everyone still uses the terms these ways.

• An ANOVA has only categorical explanatory variables and a single continuous response variable.
• A simple regression model has a singe continuous explanatory variable and a single continuous response variable.
• A multiple regression model has multiple continuous explanatory variables and a single continuous response variable.
• A regression model with a single continuous response variable, but with both categorical and continuous explanatory variables is an ANCOVA, if there are no interaction terms, or a general linear model, if there are interactions. (Nowadays, most people would say that everything mentioned above is one or another special case of the general linear model.)
• A multivariate model, say multivariate linear regression, is like the above, but it has multiple response ($Y$) variables.
• A generalized linear model, um, generalizes the general linear model, in that the response variable need not be conditionally normal. Instead, $Y$ can be distributed as any distribution that falls within the exponential family of distributions. For example, the response distribution could be binomial (yes/no) or Poisson (counts). That said, you could have either categorical explanatory variables, continuous ones, or both (and interactions are fine, too). Since the normal distribution falls within the exponential family, all of the above are considered special cases of the generalized linear model.