2
$\begingroup$

I am familiar with exponential, quadratic, power etc. transformations of variables in univariate regression when necessary based on non-normality in the relationship between the independent and dependent variable, such as skewness. My question is this: If, after reviewing each relationship via scatterplot between a dependent variable and multiple independent variables, it is determined that multiple and different transformations are needed, how does one determine the equation needed to transform the predictors? For instance, a reciprocal model may seem to fit best when comparing the DV to IV_1 and a logarithmic model when comparing DV to IV_2, but each transformation requires a different equation to back transform.

$\endgroup$
  • 1
    $\begingroup$ It's a great question, but to do justice to it requires a textbook. A very abbreviated exposition, focusing on practical, exploratory methods, is available on my Web site at quantdec.com/misc/MAT8406/Meeting07 : link to the "diagnostic plots" PDF file. In 24 heavily illustrated pages (with plenty of working R code) I provide a principled, quantitative, and effective workflow for addressing these problems. It is addressed to people learning about multiple regression at the upper undergraduate to beginning graduate level, but using minimal mathematical background. $\endgroup$ – whuber Mar 16 '16 at 20:36
  • $\begingroup$ Thank you, whuber. I have added a follow up question in the comment under Data Science Dojo's answer, if you would like to chime in there as well. $\endgroup$ – A. Vezey Mar 18 '16 at 15:20
  • $\begingroup$ If your aim's to build a predictive model on the data, setting aside some of them for this kind of exploratory analysis is a good idea. Another approach is to use polynomial or spline bases for the predictors, allowing approximation of even non-monotonic curvilinear relationships. $\endgroup$ – Scortchi - Reinstate Monica Mar 18 '16 at 15:52
  • 1
    $\begingroup$ Thanks, @Scortchi, on both accounts. I am trying to avoid a polynomial or spline-based regression because I am unable to interpret the models as clearly is if it were a standard linear regression with transformed predictors. I actually ran into this problem recently as I could not interpret the GAM I had used, which used thin-plate regression, outside of looking at the multiple R of the model. It also had poor predictive power despite having good descriptive power. If you have any insight into the follow up question I posed in a comment to Data Science Dojo, I would be very grateful. $\endgroup$ – A. Vezey Mar 18 '16 at 16:07
2
$\begingroup$

The short answer is that you apply transformations to each IV individually, creating new IV columns with linear relationships to the DV.

For example, if I have two IVs $x_1$ and $x_2$ and a DV $y$, where $x_1$ has an exponential relationship to y (i.e. $e^{x_1}~\alpha~y$) and $x_2$ has a quadratic relationship to y (i.e. $x_2^2~\alpha~y$), then I will create two new IVs $x_1'$ and $x_2'$ such that $x_1' = e^{x_1}$ and $x_2' = x_2^2$. These new IVs have a linear relationship to $y$ by definition, allowing me to use a simple linear regression model. To extract useful coefficients from the linear regression, simply apply the inverse transformation (ln for $x_1'$ and square root for $x_2'$).

The longer, more complete answer can be found at the link in whuber's comment.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ D.S.D., thank you very much for the detailed explanation. I will review whuber's website today. My follow up question is this, if you are willing: in order to back transform the predicted values to their original units of measure, do I take the inverse transformation of the coefficients in the multiple regression and then apply the model equation to the original IV's prior to their transformation? In addition, as I transform the predictors and back transform them, does the intercept require transformation? I would imagine so as the intercept would change based on the transformed variables. $\endgroup$ – A. Vezey Mar 18 '16 at 15:19
  • $\begingroup$ @A.Vezey: I couldn't quite follow this: if you've transformed the independent variables, why would you need to transform the predicted values of the dependent variable? - they're still on the original scale. $\endgroup$ – Scortchi - Reinstate Monica Mar 18 '16 at 16:04
  • $\begingroup$ @Scortchi, sorry for the confusion. Eg. in a power transformation log(y)= b0 + b1log(x), the back transformation of the predicted values are ŷ = 10^(b0 + b1log(x)). If I have one predictor that requires power transformation and another that requires a reciprocal model, in which case back transformation is ŷ = 1 / ( b0 + b1x ), how do I go about completing the back transformation of the predicted values? Does that make more sense? $\endgroup$ – A. Vezey Mar 18 '16 at 16:11
  • $\begingroup$ Yes. I think it's different enough from this question that it's worth asking separately: you're also considering transformations of the dependent variable. $\endgroup$ – Scortchi - Reinstate Monica Mar 18 '16 at 16:18
  • $\begingroup$ @Scortchi thanks for the tip, I will ask a separate question. $\endgroup$ – A. Vezey Mar 18 '16 at 16:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.