I'm reading some literature on a model which I am attempting to replicate, where the output (y) has been log transformed prior to fitting.

We must back-transform the regression-predicted outcomes to the linear scale, for further use in computation, however, this is associated with some error.

The methodology documentation states the following is used to transform back to linear scale:

$P_{ymi} = \exp({x'_{yi}\beta{ym}} + \frac{1}{2}\gamma_{ym}\sigma^{2}_{ym})$

The first term inside the $\exp$ simply represents the regressors, so no problem there. The $y$ and $m$ subscripts refer to time periods (we are fitting a separate regression for each time period) and the $i$ refers to a specific observation within the time period (the observations are being tracked over the separate time period regressions). With regard to the second term; the $\frac{1}{2}\sigma^{2}_{ym}$ appears to be Duan's Smearing Estimator to me, so just half of the variance of the residuals.

The final part of that second term confuses me... it states: $\gamma_{ym}$ is a factor that allows for the non-normality of the error terms. As this term only has the subscript $ym$, it is clearly only unique to each time-period regression; not each observation. I have fit said regressions and have indeed noticed that the residuals are non-normal. Is anyone familiar with an error term which can be used to adjust for non-normality of the residuals when back-transforming data to a linear scale? I'd greatly appreciate if you could point me in the right direction.


  • $\begingroup$ When you transform only a predictor the problem is simple. Things get interesting when you transform Y. $\endgroup$ Aug 10 at 11:56
  • $\begingroup$ @Frank Harrell Apologies, I miswrote, the output was transformed using log. The predictors are untransformed; editing now. $\endgroup$
    – Clark Kent
    Aug 10 at 12:13
  • 1
    $\begingroup$ It seems $ym$ indexes separate regressions, so why include it at all? Is there some relationship among the estimates of those regressions? Although Duan's smearing estimator is a relevant concept, it doesn't appear to be germane to the question: this is the formula for the expectation of a lognormal variable. $\endgroup$
    – whuber
    Aug 10 at 14:08
  • $\begingroup$ Yes, there is a relationship, the regressions are measuring imputed values for the same sample over different time periods. $\endgroup$
    – Clark Kent
    Aug 11 at 7:47
  • $\begingroup$ That doesn't imply there is any practical relationship among the separate estimation problems, so let's be specific: how exactly are you modeling this relationship? $\endgroup$
    – whuber
    Aug 11 at 14:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.