Linear model with log-transformed response vs. generalized linear model with log link

Question

In this paper titled "CHOOSING AMONG GENERALIZED LINEAR MODELS APPLIED TO MEDICAL DATA" the authors write:

In a generalized linear model, the mean is transformed, by the link function, instead of transforming the response itself. The two methods of transformation can lead to quite different results; for example, the mean of log-transformed responses is not the same as the logarithm of the mean response. In general, the former cannot easily be transformed to a mean response. Thus, transforming the mean often allows the results to be more easily interpreted, especially in that mean parameters remain on the same scale as the measured responses.

It appears they advise the fitting of a generalized linear model (GLM) with log link instead of a linear model (LM) with log-transformed response. I do not grasp the advantages of this approach, and it seems quite unusual to me.

My response variable looks log-normally distributed. I get similar results in terms of the coefficients and their standard errors with either approach.

Still I wonder: If a variable has a log-normal distribution, isn't the mean of the log-transformed variable preferable over the log of the mean untransformed variable, as the mean is the natural summary of a normal distribution, and the log-transformed variable is normally distributed, whereas the variable itself is not?

Answer 1

Although it may appear that the mean of the log-transformed variables is preferable (since this is how log-normal is typically parameterised), from a practical point of view, the log of the mean is typically much more useful.

This is particularly true when your model is not exactly correct, and to quote George Box: "All models are wrong, some are useful"

Suppose some quantity is log normally distributed, blood pressure say (I'm not a medic!), and we have two populations, men and women. One might hypothesise that the average blood pressure is higher in women than in men. This exactly corresponds to asking whether log of average blood pressure is higher in women than in men. It is not the same as asking whether the average of log blood pressure is higher in women that man.

Don't get confused by the text book parameterisation of a distribution - it doesn't have any "real" meaning. The log-normal distribution is parameterised by the mean of the log ($\mu_{\ln}$) because of mathematical convenience, but equally we could choose to parameterise it by its actual mean and variance

$\mu = e^{\mu_{\ln} + \sigma_{\ln}^2/2}$

$\sigma^2 = (e^{\sigma^2_{\ln}} -1)e^{2 \mu_{\ln} + \sigma_{\ln}^2}$

Obviously, doing so makes the algebra horribly complicated, but it still works and means the same thing.

Looking at the above formula, we can see an important difference between transforming the variables and transforming the mean. The log of the mean, $\ln(\mu)$, increases as $\sigma^2_{\ln}$ increases, while the mean of the log, $\mu_{\ln}$ doesn't.

This means that women could, on average, have higher blood pressure that men, even though the mean paramater of the log normal distribution ($\mu_{\ln}$) is the same, simply because the variance parameter is larger. This fact would get missed by a test that used log(Blood Pressure).

So far, we have assumed that blood pressure genuinly is log-normal. If the true distributions are not quite log normal, then transforming the data will (typically) make things even worse than above - since we won't quite know what our "mean" parameter actually means. I.e. we won't know those two equations for mean and variance I gave above are correct. Using those to transform back and forth will then introduce additional errors.

Answer 2

Here are my two cents from an advanced data analysis course I took while studying biostatistics (although I don't have any references other than my professor's notes):

It boils down to whether or not you need to address linearity and heteroscedasticity (unequal variances) in your data, or just linearity.

She notes that transforming the data affects both the linearity and variance assumptions of a model. For example, if your residuals exhibit issues with both, you could consider transforming the data, which potentially could fix both. The transformation transforms the errors and thus their variance.

In contrast, using the link function only affects the linearity assumption, not the variance. The log is taken of the mean (expected value), and thus the variance of the residuals is not affected.

In summary, if you don't have an issue with non-constant variance, she suggests using the link function over transformation, because you don't want to change your variance in that case (you're already meeting the assumption).

Answer 3

In the following I try to give some additional details to @Meg's answer with some mathematical notation.

The fixed part is the same for both, transformation and GLM. However, the transformation also affects the random part, while this is not the case for the link in the GLM.

Transformation

When we speak of a gaussian linear model with log-transformed response, we usually mean the following model

$$ \log(y) = \pmb x^T \pmb \beta + \varepsilon \qquad \text{with} \quad \varepsilon \sim N(0, \sigma^2) $$

which can also be written on the original scale of $y$ as

$$ y = \exp(\pmb x^T \pmb \beta) \exp(\varepsilon) $$

On the original scale we have

• a multiplicative error
• the error follows a $\log$-normal distribution

GLM

When we speak of a gaussian GLM with $\log$-link we usually assume the following model

$$ y \sim N(\mu, \sigma^2) \\ \log(\mu) = \pmb x^T \pmb \beta $$

which can also be written as

$$ y = \exp(\pmb x^T \pmb \beta) + \varepsilon \qquad \text{with} \quad \varepsilon \sim N(0, \sigma^2) $$

On the original scale we have

• an additive error
• the error follows a normal distribution

Answer 4

Corvus pretty much answered the question. I can add:

• Transformation introduces 'bias' such that the mean on the transformed scale is not consistent with that on the original scale (see the first formula in the answer from Corvus).
• The log-transform can be useful when effects are nonlinear and multiplicative (Pek et al. 2017). The geometric mean is equal to the exponential of the arithmetic mean of log-transformed values = exp(mu). For a log-normal distribution, the geometric mean equals the median.
• Applying the central limit theorem to the log domain, the geometric mean of a large number of independent random variables is approximately log-normally distributed around the true population geometric mean (this is sometimes called the ‘Multiplicative Central Limit Theorem’). Contrary to the answer from Corvus, the true distribution does not have to be quite log normal if the sample size is 'large enough.'

Please note that transformation of the response variable should not be used for the sole purpose of satisfying LM assumptions without consideration of changes to inference. Transformation should be guided by theory, should enhance interpretation and then estimation and interpretation should be done on the transformed scale. (Box & Cox 1964; Pek et al. 2017). Following these recommendations would limit the use of data transformation in applied statistics.

Pek, J., Wong, O. and Wong, A.C. (2017) Data transformations for inference with linear regression: clarifications and recommendations. Practical Assessment, Research and Evaluation, 22, 9. doi: https://doi.org/10.7275/2w3n-0f07

Box, G.E. & Cox, D.R. (1964). An analysis of transformations. Journal of the Royal Statistical Society: Series B, 26, 211–243.