6
$\begingroup$

In a multiple logistic regression I need to standardize one of the variables because I need to add a quadratic term. Whether I add the quadratic term as the squared original or the squared standardized, I get very similar models, same AIC. Why? The estimates of the linear term change. When I square a standardized variable, I get positive numbers after squaring half of the data (which are negative) and that should completely mess up my data. Why does it not? Can someone please explain the mathematical reason behind it and tell me if I definitely should use the squared original and if I can use the squared standardized, why does it work? I notice this is a mathematics question rather than anything but I searched and could not find anything related to this.

$\endgroup$
10
$\begingroup$

Let the variable be $x$, its mean be $\mu$, and its standard deviation $\sigma$, so that the standardized variable is $z = (x-\mu)/\sigma$. By expanding $z^2$ and collecting like powers of $x$ you can rewrite your model as

$$\eqalign{ y &= \beta_0 + \beta_1 x + \beta_2 z^2 + \varepsilon \\ &= \beta_0 + \beta_1 x + \beta_2 \left(\frac{x-\mu}{\sigma}\right)^2 + \varepsilon\\ &= \left(\beta_0 + \beta_2\frac{\mu^2}{\sigma^2}\right) + \left(\beta_1 + \beta_2\left(-2\frac{\mu}{\sigma}\right)\right)x + \left(\frac{\beta_2}{\sigma^2}\right)x^2 + \varepsilon \\ &= \alpha_0 + \alpha_1 x + \alpha_2 x^2 + \varepsilon }$$

where the $\alpha_i$ are functions of the parameters $\beta_i$ depending on the constants $\mu$ and $\sigma$ and the error terms $\varepsilon$ are the same as before. (Other multivariate regression terms would be unchanged and are not shown.) This means that one model will fit the data exactly as well as the other: any differences can be attributable to floating point rounding errors. (If they are not really tiny, you likely have huge collinearity problems.) Moreover, the parameter estimates obtained with one model can be converted to corresponding estimates with the other model, provided the estimation procedure is invariant under linear changes of parameters (which is the case with Maximum Likelihood, which is what is being used when an AIC is reported).

In short, you don't have to standardize the variable when squaring it: you get an equivalent model even when it is not standardized. Because the software will automatically standardize all variables when solving, you are free to write the model in whatever form you find most interpretable.

| cite | improve this answer | |
$\endgroup$
  • 3
    $\begingroup$ One note: You mention collinearity problems. I'd say these are pretty common when dealing with quadratic terms, especially if the original variable never gets close to 0. I've frequently seen it recommended to always mean center a variable before squaring it (it doesn't matter if you divide by the sd or not). $\endgroup$ – Peter Flom Jan 7 '13 at 20:32
  • 3
    $\begingroup$ @Peter That's a very good point. Perhaps the best way to deal with this issue is to use orthogonal polynomials--but that can create problems with interpretation. Ultimately, being able to carry out the algebraic manipulations illustrated here will give one the ability to perform numerically stable calculations and derive interpretable coefficients. $\endgroup$ – whuber Jan 7 '13 at 22:04
  • $\begingroup$ Thank you for your answers. The variable in question is indeed one that does not reach zero in this dataset (it is latitude from 22-65 degrees). I am trying to include a correlation table below Latitude Latitude Latitude_sq stLat stLat_sq Latitude 1.0000000 0.9956874 1.0000000 -0.2520726 Latitude_sq 0.9956874 1.0000000 0.9956874 -0.1612099 stLat 1.0000000 0.9956874 1.0000000 -0.2520726 stLat_sq -0.2520726 -0.1612099 -0.2520726 1.0000000 Using stLat and Latitude_sq: beta=1.7 for stLat. With stLat and stLat_sq that beta is0.22 $\endgroup$ – Zsuzsa Jan 8 '13 at 17:32
  • $\begingroup$ Sorry, try again: The pearson corr coeff between the original latitude variable and the squared standardized latitude is -0.2520726, same as between the standardized latitude variable and its squared form. Between standardized Latitude and the squared original it is (shock for me): 0.9956874. I do not yet understand this, thank you in advance for some more elaboration on this problem. The beta estimates in the model are 1.69 for stLat when the original squared is used inaddition, and 0.22 when the squared standardized is used, so the first must be completely off. ?? $\endgroup$ – Zsuzsa Jan 8 '13 at 17:41
  • 1
    $\begingroup$ Although the questions in your last two comments are related to your original question, they are genuinely different and require a different explanation. (However, if you would just graph the data as a scatterplot matrix, everything should become clear.) Consider posting them as a new question: that will also give you an opportunity to post your information in a readable form. $\endgroup$ – whuber Jan 8 '13 at 18:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.