Why is GLM different than an LM with transformed variable

Question

As explained in this course handout (page 1), a linear model can be written in the form:

$$ y = \beta_1 x_{1} + \cdots + \beta_p x_{p} + \varepsilon_i,$$

where $y$ is the response variable and $x_{i}$ is the $i^{th}$ explanatory variable.

Often with the goal of meeting test assumptions, one can transform the response variable. For example we apply the log function on each $y_i$. Transforming a response variable does NOT equate to doing a GLM.

A GLM can be written in the following form (from the course handout again (page 3))

$$ g(u) = \beta_1 x_{1} + \cdots + \beta_p x_{p} + \varepsilon_i,$$

where $u$ is just another symbol for $y$ as I understand from page 2 in the course handout. $g()$ is called the link function.

I don't really understand the difference between a GLM and LM with transformed variable from the slides in the course. Can you help me with that?

Answer 1

Transforming the response prior to doing a linear regression is doing this:

$$E(g(Y)) \sim \beta_0 + \beta_1x_1 + \ldots + \beta_px_p$$

where $g$ is a given function, and we assume that $g(Y)$ has a given distribution (usually normal).

A generalised linear model is doing this:

$$g(E(Y)) \sim \beta_0 + \beta_1x_1 + \ldots + \beta_px_p$$

where $g$ is the same as before, and we assume that $Y$ has a given distribution (usually not normal).

Answer 2

I'm not sure if this will constitute a complete answer for you, but it may help break free the conceptual logjam.

There seem to be two misconceptions in your account:

1. Bear in mind that ordinary least squares (OLS--'linear') regression is a special case of the generalized linear model. Thus, when you say "[t]ransforming a response variable does NOT equate to doing a GLM", this is incorrect. Fitting a linear model or transforming the response variable and then fitting a linear model both constitute 'doing a GLM'.

2. In the standard formulation of GLMs, what you call "$u$" (which is often represented by $\mu$, but this is just a matter of preference) is the mean of the conditional response distribution at a specific location in the covariate space (i.e., $X$). Thus, when you say "where $u$ is just another symbol for $y$", this is also incorrect. In the OLS formulation, $Y$ is a random variable and/or $y_i$ is a realized value of $Y$ for observation / study unit $i$. That is, $y$ (more generically) represents data, not a parameter.

   (I don't mean to be harping on mistakes, I just suspect that these may be causing your confusion.)

3. There is also another aspect of the generalized linear model that I don't see you mentioning. That is that we specify a response distribution. In the case of OLS regression, the response distribution is Gaussian (normal) and the link function is the identity function. In the case of, say, logistic regression (which may be what people first think of when they think of GLMs), the response distribution is the Bernoulli (/ binomial) and the link function is the logit. When using transformations to ensure the assumptions for OLS are met, we are often trying to make the conditional response distribution acceptably normal. However, no such transformation will make the Bernoulli distribution acceptably normal.