6
$\begingroup$

Maybe it is a very basic question and already answered, but I could not find a clear answer.

My plots of response vs. predictors show "curved" relationship, and log-transformation can help to achieve linearity. But it helps either when I log-transform response variable or when I log-transform explanatory variables, as well. The distributions of them all are more or less close to normal.

So, what is better to transform, response or predictors, from statistical point of view? Or it doesn't matter for regression and I only need to seek to normal distributions?

(I see here that when predictors are not transformed then R2 is related to the variance of the residuals and can be trusted. Is it the only reason?)

$\endgroup$
  • $\begingroup$ since you say that there is a curved relationship between the predictor and the response, have you considered a quadratic term in your regression model? $\endgroup$ – Eric Peterson Apr 9 '13 at 2:19
  • $\begingroup$ I tried square and root transformations; but if you mean adding squares of predictors to the regression equation, then no. $\endgroup$ – nadya Apr 9 '13 at 2:28
  • $\begingroup$ yes, if you see a curve linear relationship between predictor and response, then perhaps a model that contains a quadratic term, $X^2$ in one of the predictor terms would be a better fit. $\endgroup$ – Eric Peterson Apr 9 '13 at 2:36
  • $\begingroup$ if transforming either leads to a reasonable description of the mean, I'd focus on which one did a better job of describing the variance. What are your responses and predictors measuring? $\endgroup$ – Glen_b Apr 9 '13 at 3:44
  • $\begingroup$ My predictors are signals going through a medium with attenuation and response is the distance. Physical background of all that says that the dependence is logarithmic. @Glen_b $\endgroup$ – nadya Apr 10 '13 at 5:49
6
$\begingroup$

It depends on the situation. When you have multiple independent variables, sometimes only one of them has a nonlinear relationship - in this case, transforming the dependent variable may cause problems with the other variables. In some cases, a log transform makes more substantive sense for one or the other variable. If you log transform the DV only then you are saying that arithmetic changes in the IVs relate to geometric changes in the DV. If you transform (some or all) IVs, then just the reverse. Often, variables related to income or other amounts make more sense log transformed. That is, a change in income from \$20,000 per year to \$40,000 is more like (in some sense) a change from \$200,000 to \$400,000 than a change from \$200,000 to \$220,000. If NONE of your variables can be sensibly log-transformed, it might be better to pursue some non-linear regression such as splines.

$\endgroup$
  • $\begingroup$ Thanks for the answer. All my predictors are in nonlinear relationship with the response (and in linear between each other, highly correlated). My predictors are signals going through a medium with attenuation and response is the distance. The physical background says that the dependence is logarithmic. $\endgroup$ – nadya Apr 10 '13 at 5:55
  • $\begingroup$ So it is related mainly to the sense of variables, and is not important for the regression accuracy and future prediction of values? $\endgroup$ – nadya Apr 10 '13 at 6:00
  • $\begingroup$ Sounds like the logical first try is taking log of the DV. It is related to all the issues you mention, but a regression that makes no sense is not going to be a good one, usually. $\endgroup$ – Peter Flom Apr 10 '13 at 10:02
  • $\begingroup$ Thanks! If I'm not mistaken, in the case of transforming the response, I could then back-transform (i.e. exp()) the predicted values and thus return them to the normal physical sense, right? $\endgroup$ – nadya Apr 11 '13 at 4:13
-2
$\begingroup$

Doing linear reg. its better to tranform the independent (explanatory ) variable and use the tranformation methods according to your data and cheak r-square for the model and MSE(mean square of error) for the modle if r-square is higher and MSE is mininmum you can say tranformation is appropriate..

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.