2
$\begingroup$

I am trying to understand the given explanation for the below graph. It tries to make a comparison between poission regression with similar representation of a linear model.

enter image description here

1) When fitting a linear model with log transformed response we assume that $Var[log(Y)|X]$ is constant. How it can imply that the $Var(Y|X)$ is also constant?

Note: When fitting the linear model, they have added 0.1 to avoid the issue of zeros in the log transformation. When they are back-transforming 0.1 has been removed.

$\endgroup$
4
  • 2
    $\begingroup$ 1. Re first sentence. No, for two reasons: (i) we assume they're equal for Poisson regression, not proportional; and (ii) it's the conditional variance that's proportional to the conditional mean. 2. Why are you transforming rather than fitting the Poisson regression? What are you doing with the 0's? 3. Re third paragraph. You can't; the statement is incorrect (and the assumption of constant variance in the logs is false) 4. Your final sentence is unclear. $\endgroup$
    – Glen_b
    Aug 16, 2017 at 11:22
  • 1
    $\begingroup$ There appear to be a number of mistaken premises; and the question is written in a very unclear fashion. The question should be clarified as much as possible. $\endgroup$
    – Glen_b
    Aug 16, 2017 at 11:28
  • 1
    $\begingroup$ I believe a deal of additional clarification is still possible. For example, the original final sentence remains unedited. $\endgroup$
    – Glen_b
    Aug 16, 2017 at 11:32
  • 1
    $\begingroup$ I have edited the whole post for easy of understanding. $\endgroup$
    – shani
    Aug 16, 2017 at 11:34

2 Answers 2

3
$\begingroup$

When fitting a linear model with log transformed response we assume that Var[log(Y)|X] is constant. How it can imply that the Var(Y|X) is also constant?

It's not true. It is not the case that $\text{Var}(\log(Y)|X=x)$ being constant implies $\text{Var}(Y|X=x)$ is constant:

Plot showing constant conditional variance in the logs is not constant variance on the original scale

-- in fact that's only the case if the mean is constant.

This problem appears to be caused by the original omitting to show the fact that there's conditioning on $x$ and then forgetting that it had done so.


Note, however, that the assumption of constant variance on the log scale is untrue. If you generate data from a Poisson regression model and take logs, the conditional variance is not constant. (and this is possibly what the text was trying to explain)

plot of y vs x from Poisson regression and plot of log(y+.1) vs x

Taking logs makes it close to linear over most of the range but the variance is definitely not constant in either of these two plots.

Incidentally if you want to add a constant when taking logs, something a little above 0.4 generally works very well, but I usually just say 0.5; it's easier to remember and close enough.

$\endgroup$
11
  • $\begingroup$ Can you explain it more? $\endgroup$
    – shani
    Aug 16, 2017 at 11:46
  • $\begingroup$ What part? Please ask a specific question. $\endgroup$
    – Glen_b
    Aug 16, 2017 at 11:48
  • $\begingroup$ When fitting a linear regression with a log transformed response: $log(Y) = X\beta + \epsilon$ with $\epsilon \sim N(0, \sigma^2)$ doesn't it implies that $log(Y)|X \sim N(X\beta, \sigma^2)$? $\endgroup$
    – shani
    Aug 16, 2017 at 11:52
  • $\begingroup$ Sure, but that's not what's being claimed. $\endgroup$
    – Glen_b
    Aug 16, 2017 at 11:55
  • $\begingroup$ I am bit confused with the statements given in the text. I was trying to understand it through the above conditional distribution. Will you able to explain the idea of the text in a more understandable way? $\endgroup$
    – shani
    Aug 16, 2017 at 11:57
0
$\begingroup$
require(faraway)
pmod <- glm(doctorco ~ sex + age +income + freepoor + illness + actdays
         + hscore, family=poisson, data=dvisits)

gmod <- lm(log(doctorco + 0.1) ~ sex + age + agesq + income + levyplus + 
freepoor + freerepa + illness + actdays + hscore + chcond1, 
          data=dvisits)
model <- step(gmod)
sigma_model <- deviance(model) / model$df.residual
plot(exp(predict(model) + sigma_model/2) - .1, predict(pmod, type="response"),
 xlab="linear model responses", ylab="poisson model responses")
abline(0, 1, col="red")

This is the code which generates the original scatter plot. So it seems like when they are interpreting the variance structures they have assumed a log-normal distribution for $Y$ (for linear model) and a poisson distribution for poisson regression.

For poisson: $Var(Y|X) = E(Y|X)$.

For linear model (assuming $Y$ is log-normal):
$log(Y) = X\beta + \epsilon => Var(log(Y)|X) = cons.$

$log(y_{i}) = x_{i}'\beta + \epsilon_{i}$

$y_{i} = e^{x_{i}'\beta} e^{\epsilon_{i}}$

$Var(y_{i}|x_{i}) = e^{2 x_{i}'\beta}Var(e^{\epsilon_{i}})$

Since we assumed that $\epsilon_{i}$'s are iid $N(0, \sigma^{2})$, $e^{\epsilon_{i}}$ is log-normal. Hence variance is $e^{2\sigma^2} - e^{\sigma^{2}}$ (a constant). Thereby $Var(y_{i}|x_{i})$ is a constant.

Is that make sense?

$\endgroup$
3
  • $\begingroup$ Your equation $Var(y_{i}|x_{i}) = e^{2 x_{i}'\beta}Var(e^{\epsilon_{i}})$ looks fine ... but that's definitely not constant. You made a mistake in between that line and the conclusion. $\endgroup$
    – Glen_b
    Aug 17, 2017 at 1:43
  • $\begingroup$ From your first set of plots I can see that even when the constant variance assumption is satisfied, once you take the exponential it is not going to have constant variance at the end. Oh..Is it because of the exponential term in front of the variance term? $\endgroup$
    – shani
    Aug 17, 2017 at 7:06
  • 1
    $\begingroup$ Correct. That term is the square of the mean. So the conditional standard deviation is proportional to the mean -- which is why it spreads the way it does -- the $e^\epsilon$ multiplicative noise term is essentially a percentage-variability (e.g. if $e^\epsilon=1.1$ it means the observed value is 10% above the mean and if it's 0.9 the observed value is 10% below the mean) $\endgroup$
    – Glen_b
    Aug 17, 2017 at 9:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.