5
$\begingroup$

I recently posted an article on medium where I state that a solution when we have residuals that do not follow the normal distribution is to perform non-linear transformations to the data (i.e. log transformation). Following feedback from a Statistician, he said that performing these transformations on the data causes more harm to the data than good and provided the following reasons.

Feedback from statistician

The points made in the image above seem quite sensible, however I have seen such transformations to be quite common when performing Linear Regression.

Therefore, I ask whether transformations are much more harmful to Linear Regressions model than good?

$\endgroup$
  • 4
    $\begingroup$ This partly depends on whether your goal is to understand the relationship between X and y, or to be able to predict y. If you're interested in prediction, and transformation improves (out of sample) prediction, then transformation is a good thing! $\endgroup$ – Eoin Jul 10 at 11:44
10
$\begingroup$

It's important to distinguish between transformation of the outcome variable and transformation of predictors.

Transforming the outcome variable simply because the residuals around a regression aren't normally distributed does suffer from the problems noted. You are changing the hypothesis that you're testing, leading to the associated problems with changes in interpretation, variance, confidence intervals, etc. For example, with a log transformation of the outcome you're now saying that you expect unit changes in the linear predictor to be associated with proportional rather than additive changes in the outcome. If there is reason to suspect such an association based on your knowledge of the subject matter it might be reasonable. But nonlinear relationships between the linear predictor and the outcome are better handled by generalized linear models that take both those nonlinearities and their implications for residual variance into account.

On the other hand, transformations of predictor variables to improve the relationship between the linear predictor and outcome have important roles in modeling. For example, restricted cubic splines allow regression models (including generalized linear models) to use the data directly to find a suitable transformation of a continuous predictor variable.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for your response. This aligns a lot with what i've seen from statisticians. This comes as a surprise to me as it's very common for people to perform non-linear transformations. $\endgroup$ – Kurtis Pykes Jul 10 at 12:27
  • $\begingroup$ Kurtis, there are many good reasons to transform a response variable, which is why it might seem common. See, inter alia, stats.stackexchange.com/questions/298. $\endgroup$ – whuber Jul 10 at 15:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.