18
$\begingroup$

I've obtained a multiple linear regression model in the form $$ \mathrm{log}(Y) = \beta_0 + \beta_1x_1 + \dots + \beta_4x_4 + \beta_5x_1x_2 + \dots + \beta_{10}x_3x_4 + \beta_{11}x_1^2 + \dots + \beta_{14}x_4^2 $$ from the model I've obtained some contour plots that, for a more immediate interpretation, I've back transformed in original units (using $10^{\tilde{y}}$).

Now my question is: do I need to back transform also the model parameters, the confidence intervals of the parameters themselves and the interval of prediction for new responses? How?

I think that would be nice to back transform at least the RMSEP and RMSE in order to obtain an estimate in original units of the mean error associated with the model in prediction and in fitting. How?

$\endgroup$
0

1 Answer 1

35
$\begingroup$

This is called the re-transformation problem. I'm going to make your model a little simpler to talk about it:

$\ln{Y} = \beta_0 + \beta_1X_1 + \beta_2X_2 + \beta_3X_2^2 + \epsilon$

Now, that model does not make predictions for $Y$, it makes predictions for $\ln{Y}$. It is tempting to make predictions for $Y$ by just taking predictions for $\ln{Y}$ and exponentiating them like $\hat{Y}=\exp(\widehat{\ln{Y}})$. This is wrong (i.e. biased), though:

$\begin{align} \ln{Y} &= \beta_0 + \beta_1X_1 + \beta_2X_2 +\beta_3X_2^2 + \epsilon\\ Y &= \exp(\beta_0 + \beta_1X_1 + \beta_2X_2 + \beta_3X_2^2)\exp(\epsilon)\\ \mathrm{E}\{ Y|X\} &= \exp(\beta_0 + \beta_1X_1 + \beta_2X_2 + \beta_3X_2^2)\,\mathrm{E}\{\exp(\epsilon)|X\}\\ \hat{\mathrm{E}}\{ Y|X\} &= \exp(\widehat{\ln{Y}})\,\mathrm{E}\{\exp(\epsilon)|X\} \end{align}$

The best predictor of $Y$ is its expectation. If we could conclude that $\mathrm{E}\{\exp(\epsilon)|X\}=1$, then we could just exponentiate like you are suggesting above. But Jensen's inequality says that since $\mathrm{E}\{\epsilon|X\}=0$, it must be that $\mathrm{E}\{\exp(\epsilon)|X\}>1$. So, we have to use some kind of adjustment. The adjustment is called Duan's Smearing Estimator. It is just the sample mean of the exponentiated prediction errors (residuals) from the original model, $(1/N)\sum \exp(e_i)$. So the right way to re-transform from the log model back to predictions on Y is:

$$\hat{Y}_j = \exp(\widehat{\ln{Y}}_j) \cdot \frac{1}{N}\sum_{i=1}^N \exp(e_i)$$

To your questions. On the parameters, whether you need to re-transform depends on what you are trying to measure. The parameter $\beta_2$ measures the amount that $Y$ goes up (in percents) for a one unit increase in $X_1$. So, if $\beta_2=0.04$, that says that $Y$ goes up 4% for each one unit $X_1$ goes up. Similarly, for each unit $X_2$ goes up, $Y$ goes up $\beta_2+2\beta_3X_2$ percent.

If you want to measure the amount that $Y$ goes up in units when $X_2$ goes up by one unit, then you need to re-transform:

$\begin{align} Y &= \exp(\beta_0 + \beta_1X_1 + \beta_2X_2 + \beta_3X_2^2)\exp(\epsilon)\\ \frac{\partial Y}{\partial X_1} &= \exp(\beta_0 + \beta_1X_1 + \beta_2X_2 + \beta_3X_2^2)\exp(\epsilon)\beta_1\\ \hat{\mathrm{E}}\left\{ \frac{\partial Y}{\partial X_1} \right\} &= \exp(\widehat{\ln{Y}}) \cdot \frac{1}{N}\sum \exp(e_i) \cdot \beta_1 \end{align}$

Notice that the answer depends on $\hat{Y}$, totally unlike "regular" regression. You should expect this, though. The model is non-linear, so the derivative depends on the point of evaluation. For your more complicated model, you have to be careful to apply the chain rule properly---that is, where I have $\beta_1$, you will have a complicated expression with $\beta$s and powers of your various $X$s and such.

For the confidence intervals, again, the question is what you are trying to measure. If you are happy with knowing how many percents $Y$ goes up when $X_1$ goes up by one, then the "regular" confidence intervals you get from the usual regression output are fine. If you want to measure the number of units that $Y$ goes up when $X_1$ goes up by one, then it's more complicated. Actually, it's very complicated in that case---you should use bootstrapping to do it. You can use something called the delta method, but it is a pain.

Root mean squared error of the prediction is easy to calculate, once you have re-transformed back to predicted $Y$:

$$\mathrm{RMSEP} = \sqrt {\frac{1}{N-1} \sum (\hat{Y}_i-Y_i)^2}$$

where $\hat{Y}_i$ comes from the formula above.

$\endgroup$
8
  • $\begingroup$ Thank you very much! The problem is really more complicated than I thought. $\endgroup$
    – Ndr
    Commented May 7, 2013 at 8:51
  • $\begingroup$ @Bill Sorry to post onto this thread so long after the initial question, but why do we take the expectation of Y in the third equation given above? Couldn't we say that $\exp(\epsilon) = 1$ when $\epsilon = 0$, and $\exp(\epsilon) > 1$ when $\epsilon > 0$? Although, this doesn't account for when $\epsilon < 0$ which would surely mean that $0 < \exp(\epsilon) < 1$? I may be missing something, but why do we take the expectation to prove this? Is it required? $\endgroup$
    – sym246
    Commented Nov 24, 2017 at 12:31
  • $\begingroup$ @sym246 The best forecast of a random variable (best in the sense of minimizing mean squared error) is its expectation conditional on whatever random variables we know at the time we are making the forecast. So, if you want to forecast $Y$, then the best choice is its expectation given what we know, and that is $X$ under the assumption of the classical linear regression model. $\endgroup$
    – Bill
    Commented Nov 25, 2017 at 13:26
  • $\begingroup$ @Bill if my understanding is correct, this is applicable for regression model only right? There is no need to apply the same logic on decision trees, random forests etc? $\endgroup$ Commented Jan 7, 2019 at 11:56
  • 1
    $\begingroup$ @Amonet Yes. I should not have said "original model." It would have been better to say "estimated model." You take the residuals from the estimated model (with lnY as the left-hand-side), exponentiate them, and take their sample mean. Best, Bill. $\endgroup$
    – Bill
    Commented Feb 9, 2019 at 15:19

Not the answer you're looking for? Browse other questions tagged or ask your own question.