1
$\begingroup$

Trying to understand how log-transforming a count variable changes the regression equation.

The reason I ask is because I see a lot of researchers log-transform their data so they can use OLS.

Say you have a Poisson GLM:

$$ y = exp(\alpha + \beta x) \implies log(y) = \alpha + \beta x $$

where $y$ is a Poisson random variable with zeros.

Suppose you want to log-transform it with a constant $c$. Does that mean your equation becomes

$$ log(y + c) = \alpha + \beta x + c $$

?

$\endgroup$
4
  • 2
    $\begingroup$ Could you please edit the question to explain a situation in which you would want to add a constant to your outcome measurement this way? That would seem to undercut the assumption underlying the Poisson GLM. $\endgroup$
    – EdM
    Jul 23, 2022 at 21:33
  • 3
    $\begingroup$ This is unclear, you would be better off asking about your real problem. But please, state trhe real problem in ordinary, non-technical english. But have a look at stats.stackexchange.com/questions/8505/…, stats.stackexchange.com/questions/47840/… $\endgroup$ Jul 23, 2022 at 22:13
  • $\begingroup$ @EdM edited with my motivation. $\endgroup$
    – invictus
    Jul 23, 2022 at 22:29
  • 8
    $\begingroup$ That isn't the relevant GLM equation; the GLM equation is $\mathbb{E}y = \exp(\alpha + \beta x)$ (actually, $\log \mathbb{E}y = \alpha + \beta x$.) There's a huge difference; taking the log of $y$ isn't equivalent to running a GLM. $\endgroup$
    – jbowman
    Jul 23, 2022 at 23:01

1 Answer 1

4
$\begingroup$

As comments and their links note, there's a difference between a Poisson GLM with a log link and ordinary least squares (OLS) linear regression with a log-transformed count outcome. The Poisson GLM models the log of the expected value of the counts, based on the equality of mean and variance for a Poisson-distributed variable. OLS with log-transformed counts models the expected value of the log of the counts and handles residual error as in OLS, but in the log-transformed scale.

When you add a constant to the observed counts to avoid taking the log of 0, you are changing the effective outcome variable. You are not adding a constant offset to the right-hand side of the equation, as your last equation would seem to imply. If you add, say, 0.5 to a count value of 1, then the difference between log(1.5) and log(1) is about 0.4. If you add 0.5 to a count value of 100, then the difference between log(100.5) and log(100) is about 0.01. That's not a constant offset.

This page discusses ways to handle 0 counts in regression. The square-root transformation doesn't require adding anything to the observed counts, and approximately stabilizes the variance for a Poisson-distributed variable. For linear regression I tend to use square-root for counts, reserving log transformations for necessarily positive values whose measurement error is proportional to the mean.

As @Glen_b explains on the page linked above, the different approaches of a Poisson GLM and linear regression have practical implications:

The canonical [log] link is not generally a particularly good transformation for Poisson data... It's a good 'transformation' for the conditional population mean of a Poisson in a number of contexts, but not always of Poisson data.

He goes on to note that adding a constant of about 0.5 to outcome counts can work well if you nevertheless want to log transform. But other than changing the outcome values somewhat so you avoid 0 counts, it's not a transformation of the underlying regression equation in the way you suggest.

You can see this with the simplest linear model, of the mean value. Set up two large populations of Poisson-distributed values with means of 1 or 4, compare the means of the original values against exp(mean(log(y+0.5))) for 1000 samples of 100 cases each.

set.seed(101)
y1 <- rpois(100000,1)
y4 <- rpois(100000,4)
y1c <- y1+0.5 ## for c = 0.5
y4c <- y4+0.5 ## for c = 0.5
mean(y1)
# [1] 0.99968
mean(y1c)
# [1] 1.49968
mean(y4)
# [1] 4.00483
mean(y4c)
# [1] 4.50483
meanExp1 <- NULL
for (i in 1:1000) meanExp1 <- c(meanExp1,exp(mean(log(sample(y1c,100)))))
mean(meanExp1)
# [1] 1.187415
meanExp4 <- NULL
for (i in 1:1000) meanExp4 <- c(meanExp4,exp(mean(log(sample(y4c,100)))))
mean(meanExp4)
# [1] 4.0049

The negative bias from the y + 0.5 values is obvious, and different depending on the initial distribution of y.

Poisson GLM models of the original count values perform as expected.

meanPoisGLM1 <- NULL
for (i in 1:1000) meanPoisGLM1 <- 
  c(meanPoisGLM1,exp(coef(glm(sample(y1,100)~1,family=poisson))[[1]]))
mean(meanPoisGLM1)
# [1] 0.99892
meanPoisGLM4 <- NULL
for ( i in 1:1000) meanPoisGLM4 <- 
  c(meanPoisGLM4,exp(coef(glm(sample(y4,100)~1,family=poisson))[[1]]))     
mean(meanPoisGLM4)
# [1] 3.99786
$\endgroup$
8
  • $\begingroup$ yeah i get that there is a whole debate about transforming count variables. i just wanted try writing down the equations. so the glm for a poisson rv is $log(\mathbb{E}[y]) = \alpha + \beta x$. now say you add a constant $c$. Does that mean you are estimating $log(\mathbb{E}[y+c]) = \alpha + \beta x$? $\endgroup$
    – invictus
    Jul 24, 2022 at 19:48
  • $\begingroup$ or are you actually estimating $\mathbb{E}[log(y+c)] = \alpha + \beta x$? $\endgroup$
    – invictus
    Jul 24, 2022 at 20:03
  • 1
    $\begingroup$ @invictus it depends whether you are doing GLM or OLS. GLM would try to do what's in your first comment, OLS would do what's in the second. I emphasize try because if $y$ were originally Poisson, $y+c$ wouldn't be. That's obvious if you choose $c=0.5$, a pretty good choice for OLS. $\endgroup$
    – EdM
    Jul 24, 2022 at 20:32
  • 1
    $\begingroup$ @invictus it's the link function that matters. That's distinguished from the distribution family (Poisson, Gaussian, etc) used in the GLM. The choice of family includes an assumption about the mean/variance relationship; they are independent with a Gaussian GLM. A Gaussian GLM with identity link is equivalent to OLS. I suppose you could do a Gaussian GLM with a log link on count data. Then you would have a model like in your first comment, but with a different assumption than the Poisson GLM about the error term. $\endgroup$
    – EdM
    Jul 25, 2022 at 14:50
  • 1
    $\begingroup$ @invictus I added an example of the bias to the end of the answer. Simple simulations like that can show a lot. $\endgroup$
    – EdM
    Jul 25, 2022 at 16:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.