Linear model vs general linear model

What's the difference between a linear model and a general linear model?

Maybe: $Y=a X + b$ - general linear and $Y=a \psi(X) + b$ - linear ?

These forms come from the relevant Wikipedia articles on the linear and general linear models.

• Do you have a specific context in mind? The terms are often used interchangeably Commented Sep 6, 2016 at 14:53
• What would make your first formula "general"? I assume you have reversed the labels from your intent. Commented Sep 6, 2016 at 15:00
• You are missing some elements from Wikipedia which (a) defines general as being about a possibly multivariate outcome (b) allows for the X to receive a transformation in the linear definition but for some reason not in the other one. Commented Sep 6, 2016 at 15:34
• If you quote wikipedia, you should normally actually quote it and link to the sections the quote comes from (or at the least, link to the sections you're discussing even if you don't quote). Note that different articles are contributed to by different people and they don't always have the same take on the same topic. Commented Sep 6, 2016 at 15:41
• I put your links in the question (they should be there rather than in comments). I also attempted to add the general-linear-model tag but it remaps to multiple-regression (this is unfortunate, to my mind because the general linear model is more general than multiple regression as usually written). Commented Sep 6, 2016 at 15:52

A general linear model doesn't generalize the function of $X$.

Indeed assuming you mean $E(Y|X)$ where you have $Y$ (and independent errors) whether or not there's a transformed predictor doesn't change things -- either way it would still be called a linear model (the conditional mean is a linear function of the parameters).

That is to say, consider $\alpha+\beta \psi(X)$. Now let $X^* = \psi(X)$. Then in terms of this new variable (the one used in the estimation) we have $\alpha+\beta X^*$. So either a linear model or a general linear model will be able to incorporate a transformation, $\psi$, (of the independent variable or variables) without difficulty.

Instead, with a multivariate response (each observation point is a vector of values), a general linear model generalizes the covariance structure of the error term so that the response values includes the possibility of correlated errors within the observation vector (that is, the components of $\mathbf{y}_i$ are correlated).

This feature allows us to place under one banner t-tests, ANOVA, regression, MANOVA, MANCOVA, multivariate regression and a number of other models/tools (while the multivariate techniques wouldn't necessarily be seen as covered by the term 'linear model', though the usage does vary).

[Between observations there is still independence; if you want instead to generalize to correlated errors between-observations, that would be generalized least squares as fcop pointed out in comments.]

• could it be that what you describe is ''general least squares'' ? see e.g. data.princeton.edu/wws509/notes/a2.pdf versus homepage.ntu.edu.tw/~ckuan/pdf/et01/et_Ch4.pdf
– user83346
Commented Sep 6, 2016 at 19:21
• @fcop see the first paragraph of the wikipedia article on general linear models I linked, in particular the part where it talks about the errors $U$ coming from a multivariate normal distribution. However, because of the fact that we have two potential sources of correlation (within the observation vector and between observation vectors) my post was ambiguous/unclear/misleading. I will clarify it to make it clearer that it's the first thing. Thanks for the helpful comment. Commented Sep 6, 2016 at 22:50

In a linear model, we define prediction or regression function using a linear structure as follows: $$y\approx E(y|x)=\omega_0 + \omega^\top x.$$

While in a generalized linear model, we define prediction function or discriminatory function either as a linear in parameter or a non-linear in parameter through linear argument ($$\omega^\top x +\omega_0$$).

That is the hypothesis function for generalized linear model is $$h(x)=g(\omega^\top x +\omega_0)$$, where g may be a linear or non-linear function (known as activation function). While estimating the hypothesis function, we focus on estimating the parameter $$\omega$$ only as g is defined as per requirment.