I have the following dependent and independent variables for my linear regression model. Since they are all in different scales (some of the are % others continuous variables), I was suggested to take the log and normalize my variables before running the regression.

    Y    X2        X3 (%)       X1 (%)
Mean 2.9 24.6   0.009517    230.992248
std  2.3 32.2   0.077092    230.992248
Min  0   1      0           0
Max  8   539    1           1

I have the following Qs:

Why should I take the log and then normalize them - rather than using just one of the two data transformations?

Should I log and normalize also my Y variable?

How would interpret my coefficient at the end of the exercise? and how can I make them humanly intelligible for a business audience?

Any easy doc reference is very appreciated!

  • $\begingroup$ I don't know why you would do that, doesn't really make sense to me, and good luck interpreting that. Was there a reason given for this recommendation? Some domain knowledge behind this statement? $\endgroup$ – user2974951 Jan 11 at 7:23
  • $\begingroup$ Good point, not sure there was. I think the suggestion came from two factors: X1 is not normally distributed (but is not a necessary condition for OLS, right?) so log normalization would have helped to normalize it. Second point, to ensure comparability between regressors - but that it is not really necessary for me. I was wondering whether it was incorrect to leave the data as it is.. $\endgroup$ – Filippo Sebastio Jan 11 at 7:28
  • $\begingroup$ Forgive me if I misunderstood but I have to warn you that normalization is not the process of making a variable normally distributed. $\endgroup$ – gunes Jan 11 at 7:55
  • $\begingroup$ Apologies, it was not clear from my message, what I meant is that my colleague suggested taking the log (not to normalize) of the continuous variable to make it a normal distribution (currently it is not). $\endgroup$ – Filippo Sebastio Jan 11 at 8:04

First of all, taking only the logarithm of predictors creates the regression equation $y=a+\sum w_i\log x_i$, which assumes a relationship in the form $e^y=A\prod{x_i^{w_i}}$. Taking log of both IVs and DV creates the problem $\log y=a+\sum{w_i\log x_i}$, which assumes a relation in the form $y=A\prod{x_i^{w_i}}$. So, the underlying assumption changes. It's not that you must take the logarithm of $y$, it's your underlying assumption. And if you're unsure which relation is actually meaningful, you need to do model selection and test both.

In my opinion, standardization is a common step in regression analysis. Once you set your predictors, i.e. $\hat{x_i}=\log x_i$, you'll solve for the regression problem $y=\sum{w_i\hat{x_i}}+b$. The thing is, when you actually try to solve for $y=\sum{w_ix_i}+b$, you'd probably apply standardization without questioning. U think, doing the same for $\hat{x_i}$ here would also do no harm. The purpose of standardization is to ensure that each term contributes to MSE similar amounts. And you don't need to apply standardization to $y$, however applying doesn't also harm.

Note: some of your variables contains $0$'s (or numbers close to $0$) I guess. Be careful when you take the logarithm.


Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.