5
$\begingroup$

In many Kaggle competitions where linear regression has been applied, I see people plot the y distribution and then take the log of (or other transformation of) the dependent variable to make y normal distributed. What would happen if we don't do this step: transform the dependent variable into a normal distribution?

I have a few thoughts but wasn't sure which one is more correct and which one is wrong.

  1. if y is skewed, after running the linear regression model, we could have the heteroscedasticity problem (variance is not constant along the predicted y)
  2. we know the linear regression coefficients are still unbiased even if y is skewed; however, the t-test of the coefficient wouldn't make sense anymore because y is not a normal distribution.
  3. Because X is usually standardized, y's distribution should match X's normal distribution.
Improve this question
$\endgroup$
7
  • $\begingroup$ Is that for the regression or just the graph? $\endgroup$
    – Dave
    Commented Sep 28, 2022 at 4:12
  • 5
    $\begingroup$ taking log doesn't make a distribution normal $\endgroup$
    – Aksakal
    Commented Sep 28, 2022 at 5:14
  • $\begingroup$ Your title (but not the question body) also asks about standardising the independent variables, which I do not address in my answer. That's probably already been asked & answered on this site, but if not, I suggest posting that as a separate question. $\endgroup$
    – mkt
    Commented Sep 28, 2022 at 10:36
  • 2
    $\begingroup$ @Galen, standardizing non-normal variables can never make them normal, as the family of Normal distributions is closed under changes of location and scale. $\endgroup$
    – Richard Hardy
    Commented Sep 28, 2022 at 17:04
  • $\begingroup$ @RichardHardy Agreed. I could have stated the problem more strongly. My comment alludes to existence of counterexamples, but we can use the universal quantifier here. $\endgroup$
    – Galen
    Commented Sep 28, 2022 at 17:06

1 Answer 1

Reset to default
19
$\begingroup$

All 3 of your points are incorrect.

The log transformation does not necessarily make the dependent variable normally distributed. But more importantly, it doesn't matter if the dependent variable is normally distributed or not - or the independent variable either. Instead, what we typically care about is that the residuals of the fitted model are normally distributed, and even that is a helpful feature, not a strict requirement. Also, the residuals can be normally distributed even when the predictor and response are not.

You may find these threads helpful:

When (and why) should you take the log of a distribution (of numbers)?

Interpretation of log transformed predictor and/or response

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.