11
$\begingroup$

I'm fitting a regression on the $\log(y)$. Is it valid to back transform point estimates (and confidence/prediction intervals) by exponentiation? I don't believe so, since $E[f(X)] \ne f(E[X])$ but wanted other's opinions.

My example below shows conflicts with back transforming (.239 vs .219).

set.seed(123)

a=-5
b=2

x=runif(100,0,1)
y=exp(a*x+b+rnorm(100,0,.2))
# plot(x,y)

### NLS Fit
f <- function(x,a,b) {exp(a*x+b)} 
fit <- nls(y ~ exp(a*x+b),  start = c(a=-10, b=15)) 
co=coef(fit)
# curve(f(x=x, a=co[1], b=co[2]), add = TRUE,col=2,lwd=1.2) 
predict(fit,newdata=data.frame(x=.7))
[1] 0.2393773

### LM Fit
# plot(x,log(y))
# abline(lm(log(y)~x),col=2)
fit=lm(log(y)~x)
temp=predict(fit,newdata=data.frame(x=.7),interval='prediction')
exp(temp)
        fit       lwr       upr
1 0.2199471 0.1492762 0.3240752
$\endgroup$
  • 1
    $\begingroup$ Is this not one of the problems that is solved by log-linked gaussian GLMs? $\endgroup$ – generic_user Sep 16 '14 at 5:31
  • $\begingroup$ @ARM Yes I believe so. Thanks for pointing that out. However using GLM it is harder to get prediction intervals but I think I can work it out. $\endgroup$ – Glen Sep 16 '14 at 6:14
  • 1
    $\begingroup$ @Glen Do a search for Duan smearing on this site. $\endgroup$ – Dimitriy V. Masterov Sep 16 '14 at 17:32
13
$\begingroup$

It depends on what you want to obtain at the other end.

A confidence interval for a transformed parameter transforms just fine. If it has the nominal coverage on the log scale it will have the same coverage back on the original scale, because of the monotonicity of the transformation.

A prediction interval for a future observation also transforms just fine.

An interval for a mean on the log scale will not generally be a suitable interval for the mean on the original scale.

However, sometimes you can either exactly or approximately produce a reasonable estimate for the mean on the original scale from the model on the log scale.

However, care is required or you might end up producing estimates that have somewhat surprising properties (it's possible to produce estimates that don't themselves have a population mean for example; this isn't everyone's idea of a good thing).

So for example, in the lognormal case, when you exponentiate back, you have a nice estimate of $\exp(\mu_i)$, and you might note that the population mean is $\exp(\mu_i+\frac{1}{2}\sigma^2)$, so you may think to improve $\exp(\hat{\mu_i})$ by scaling it by some estimate of $\exp(\frac{1}{2}\sigma^2)$.

One should at least be able to get consistent estimation and indeed some distributional asymptotics via Slutsky's theorem (specifically the product-form) as long as one can consistently estimate the adjustment. The continuous mapping theorem says that you can if you can estimate $\sigma^2$ consistently... which is the case.

So as long as $\hat{\sigma}^2$ is a consistent estimator of $\sigma^2$, then $\exp(\hat{\mu_i})\cdot \exp(\frac{1}{2}\hat{\sigma}^2)$ converges in distribution to the distribution of $\exp(\hat{\mu_i})\cdot \exp(\frac{1}{2}\sigma^2)$ (which by inspection will then be asymptotically lognormally distributed). Since $\hat{\mu_i}$ will be consistent for $\mu_i$, bu the continuous mapping theorem, $\exp(\hat{\mu_i})$ will be consistent for $\exp(\mu_i)$, and so we have a consistent estimator of the mean on the original scale.

See here.

Some related posts:

Back transformation of an MLR model

Back Transformation

Back-transformed confidence intervals

$\endgroup$
  • 1
    $\begingroup$ Thanks, I looked at the previous posts and, while enlightening, was still somewhat confused, hence my question. $\endgroup$ – Glen Sep 16 '14 at 6:15
  • $\begingroup$ +1 Great answer! Just a quick clarification: Where did the $\frac{1}{2}$ came from as a scaler for $\hat{\sigma^2}$? I saw it in the definition of the lognormal in Wikipedia but it is not explained there either, is it just integrating out mean from the PDF? $\endgroup$ – usεr11852 says Reinstate Monic Jul 15 at 23:55
  • 1
    $\begingroup$ You should be able to get it just by directly integrating: $E(Y) = \int_0^\infty y\, f(y)\, dy$ where $f$ is the density for the lognormal, but it is probably easier to do by calculating $E(e^X)$ for a normal (where $X=\log Y$), but then perhaps it is better to find the MGF for $X$ - which is no more difficult - and from which moments for $Y$ are very readily obtained (by replacing $t$ by $1,2,...$ in turn), essentially getting higher moments for free. $\endgroup$ – Glen_b -Reinstate Monica Jul 16 at 1:19
  • 1
    $\begingroup$ @usεr11852 In either of the latter cases you take the $e^x$ or $e^{tx}$ into the $e^{...}$ term in the density, then complete the square in $x$, and bring additional constants (i.e. all except the normalizing constant for the normal) out the front of the integral (which has the $\frac12$ in it), leaving a Gaussian pdf integrated on the real line (with shifted mean from the original) which integrates to 1, leaving only the constants you brought out the front. This involves nothing more than very simple algebraic manipulations, ... ctd $\endgroup$ – Glen_b -Reinstate Monica Jul 16 at 2:12
  • 1
    $\begingroup$ ctd... and from which the $t$-th raw moment of a lognormal is $e^{\mu t + \frac12 \sigma^2t^2}$. $\endgroup$ – Glen_b -Reinstate Monica Jul 16 at 2:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.