I need to automate the transformation on some linear regression models. There is only one predictor in this case. Sometimes i get a good model with the original variables, sometimes i need to log the predictor, and in some cases log both sides.

I'm using R, so what kind of tests/packages can i use to automate this? I'm using Pearson correlation now, but i'm not sure if it makes sense.


PS: This may look a duplicate question, but i couldn't find yet the methodology to apply.

  • $\begingroup$ Look into Box-Cox transformations, that should be at least a good start. $\endgroup$ – Peter Flom - Reinstate Monica Jan 29 '13 at 13:33
  • $\begingroup$ @PeterFlom I think the OP is looking for some search algorithm that looks to return a normal distribution from the original data. I've always done this by hand visually inspecting the transformation result, the question is quite interesting. $\endgroup$ – Robert Kubrick Jan 29 '13 at 16:44
  • 1
    $\begingroup$ The problem is that i have a separate model for each region of my country, so i can't inspect one by one. There must be some kind of test or package to automate this! $\endgroup$ – Fernando Jan 29 '13 at 18:24

I'm not sure this is a good idea. By having different transformations for different regions you are fundamentally changing the relationship between the variables. Is this really what you want? Might you be better off with some kind of single integrated model, perhaps a mixed effects one with a region random effect?

But if you want to, it must be reasonably straightforward to write code to do it. Write a function that takes in a vector of data, performs each of the candidate transformations, does a test for normality eg the Shapiro-Wilk test on each of the results, selects the one with the highest p value (ie the highest probability that the resulting data has come from a normal distribution), and returns a list with two elements - a vector of the transformed data, and an object holding some extra info on which transformation was used.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.