Suppose that you are have a response variable $Y$ and explanatory variables $X_1$, $X_2$ and $X_3$. If we want to use a quadratic transformation for $X_1$, would we still include $X_1$? In other words, would we have:

$$E[Y|X] = \beta_0+\beta_{1}X_{1}^{2} + \beta_{2}X_{2} + \beta_{3}X_{3}$$ or $$E[Y|X] = \beta_0+\beta_{1}X_{1} + \beta_{2}X_{1}^{2} + \beta_{3}X_{2}+\beta_{4}X_{3}$$

If instead we did a logarithmic transformation, then would it just be:

$$E[Y|X] = \beta_0+\beta_{1} \log(X_1) + \beta_{2}X_{2} + \beta_{3}X_{3}$$

  • $\begingroup$ NB the first model imposes a constraint that the minimum or maximum of $Y$ plotted against $X_1$ occurs at exactly $X_1=0$. You'll want to be sure that's sensible. It often isn't, & therefore the first model is ruled out a priori. $\endgroup$ Mar 11 '14 at 9:50

Either model could work, and which to use depends on why you transformed. If you took the log because $Y$ is expected to be linearly related to $\log X_1$ (e.g. response to log dose of medication), then you would probably just go with $\log X_1$ and drop the untransformed term. On the other hand if you added a quadratic term because it looked like $Y$ was related to $X_1$ in a polynomial fashion (e.g. crop yield vs latitude) you would probably try keeping both $X_1$ and $X_1^2$ and then drop one (or both) if it turned out to not be significant.

That being said, most people keep both terms unless there's an obvious reason not to and then use a model selection algorithm to decide which to keep.

  • $\begingroup$ @P Schnell: In confirmatory analysis you would determine a priori what terms to keep? $\endgroup$ Mar 11 '14 at 1:16
  • $\begingroup$ Assuming that you mean confirming that a given model fits a fresh sample from the population it's supposed to describe, then yes. Of course, your transformations should then also be specified a priori as well. $\endgroup$
    – P Schnell
    Mar 11 '14 at 1:31
  • $\begingroup$ @P Schnell: Is there a good way of knowing what transformations to consider without using any exploratory data analysis? $\endgroup$ Mar 11 '14 at 1:35
  • $\begingroup$ Prior knowledge about the system and/or a priori constraints on the model (e.g. in confirmatory factor analysis). Usually though "we'll look at some plots and transform the predictors as necessary" is a perfectly acceptable procedure, and if it's not I'd expect someone involved in the project would have a very strong and/or informed opinion about what transformations should be used anyway. $\endgroup$
    – P Schnell
    Mar 11 '14 at 2:12
  • $\begingroup$ @P Schnell: Would you say that higher order terms signify that a variable is important? $\endgroup$ Mar 11 '14 at 2:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.