9
$\begingroup$

I want to include the term $x$ and its square $x^2$ (predictor variables) into a regression because I assume that low values of $x$ have a positive effect on the dependent variable and high values have a negative effect. The $x^2$ should capture the effect of the higher values. I therefore expect that the coefficient of $x$ will be positive and the coefficient of $x^2$ will be negative. Besides $x$, I also include other predictor variables.

I read in some posts here that it is a good idea to center the variables in this case to avoid multicollinearity. When conducting multiple regression, when should you center your predictor variables & when should you standardize them?

  1. Should I center both variables seperately (at the mean) or should I only center $x$ and then take the square or should I only center $x^2$ and include the original $x$?

  2. Is it a problem if $x$ is a count variable?

In order to avoid $x$ being a count variable, I thought about dividing it by a theoretically defined area, for example 5 square kilometers. This should be a little bit similar to a point density calculation.

However, I am afraid that in this situation my initial assumption about the sign of the coefficients would not hold anymore, as when $x=2$ and $x²=4$

$x= 2 / 5 \text{ km}^2$ = $0.4 \text{ km}^2$

but $x^2$ would then be smaller because $x^2= (2/5)^2= 0.16$.

$\endgroup$
1
  • 1
    $\begingroup$ Your regression software will take care of numerical issues automatically--in particular, it is highly likely to center and standardize your data internally. How to answer your questions about centering comes down to how you wish to interpret the coefficients. $\endgroup$
    – whuber
    May 23, 2014 at 15:40

3 Answers 3

5
$\begingroup$

Your question is in fact comprised of several sub-questions, which I will try to address to the best of my understanding.

  • How to distinguish low and high values' dependence on a regression?

Considering $x$ and $x^2$ is a way of doing it, but are you sure your test is conclusive? Will you be able to conclude something useful for all possible outcomes of the regression? I think posing the question clearly beforehand can help, and asking similar and related questions can help as well. For instance, you can consider a threshold of $x$ for which the regression slopes are different. This can be done using moderator variables. If the different slopes (while imposing the same intercept) are compatible then you have no difference, otherwise you provided yourself a clear argument for their difference.

  • When should you center and standartize?

I think this question should not be mixed with the first question and test, and I'm afraid centering around $x$ or $x^2$ beforehand might bias the results. I would advise not to center, at least in a first stage. Remember you will probably not die of multicollinearity, many authors argue it's just equivalent to working with a smaller sample size (here and here).

  • Does transforming the discrete count variable in a (continuous) floating-point variable change the interpretation of the results?

Yes it will, but this will depend heavily on the first 2 points, so I would suggest you to address one thing at a time. I see no reason why the regression would not work without this transformation, so I would advise you to ignore it for now. Note also that by dividing by a common element you are changing the scale at which $x^2 = x$, but there are completely different ways of looking at it, like I wrote above, in which this threshold is considered in more explicit way.

$\endgroup$
2
  • $\begingroup$ Thank you very much for your answer, especially for the links!!! $\endgroup$
    – Peter
    May 25, 2014 at 12:03
  • $\begingroup$ It was a pleasure to help. =) $\endgroup$ May 25, 2014 at 14:32
4
$\begingroup$

In general centering could help to reduce multicollinearity, but "you will probably not die of multicollinearity" (see predrofigueira's answer).

Most important, centering is often needed to make the intercept meaningful. In the simple model $y_i=\alpha+\beta x_i+\varepsilon$, the intercept is defined as the expected outcome for $x=0$. If an $x$ value of zero is not meaningful, neither the itercept is. It is often useful to center the variable $x$ around its mean; in this case, the predictor is of the form $(x_i-\bar{x})$ and the intercept $\alpha$ is the expected outcome for a subject whose value on $x_i$ is equal to the mean $\bar{x}$.

In such cases, you must center $x$ and then square. You cannot center $x$ and $x^2$ separately, because you are regressing the outcome on a "new" variable, $(x_i-\bar{x})$, so you must square this new variable. What could centering $x^2$ mean?

You can center a count variable, if its mean is meaningful, but you could just scale it. For example, if $x=1,2,3,4,5$ and "2" could be a baseline, you can subtract 2: $(x_i-2)=-1,0,1,2,3$. The intercept becomes the expected outcome for a subject whose value on $x_i$ is equal to "2", a reference value.

As to dividing, no trouble: your estimated coefficients would be larger! Gelman and Hill, §4.1, give an example: $$\begin{align} \text{earnings}&=-61000+1300\cdot\text{height (in inches)}+\text{error} \\ \text{earnings}&=-61000+51\cdot\text{height (in millimeters)}+\text{error}\\ \text{earnings}&=-61000+81000000\cdot\text{height (in miles)}+\text{error} \end{align}$$

One inch is $25.4$ millimeters, so $51$ is $1300/25.4$. One inch is $1.6e-5$ emiles, so $81000000$ is $1300/1.6e-5$. But these three equations are entirely equivalent.

$\endgroup$
3
  • $\begingroup$ related. $\endgroup$
    – Henrik
    May 24, 2014 at 12:07
  • $\begingroup$ Thanks for your answer Sergio. It really helped me. Unfortunately I can only mark one answer as my accepted answer. $\endgroup$
    – Peter
    May 25, 2014 at 11:47
  • $\begingroup$ You're welcome. And don't worry ;-) $\endgroup$
    – Sergio
    May 25, 2014 at 13:58
1
$\begingroup$

I assume that low values of x have a positive effect on the dependent variable and high values have a negative effect.

While I appreciate others' treatment of centering and interpretation of coefficients, what you've described here is simply a linear effect. In other words, what you've described doesn't indicate any need to test the square of x.

$\endgroup$
3
  • $\begingroup$ In my view, if $y=\beta_0+\beta_1 x_1+\beta_2 x_2+\varepsilon$, the (partial) effect of $x_i$ on $y$ (or, better, on $E[y\mid \mathbf{x}]$) is $\partial E[y\mid \mathbf{x}]/\partial x_i=\beta_i$. Such effects are constant, they do not depend on the level of $x_i$. If the model is $y=\beta_0+\beta_1 x_1+\beta_2 x_2+\beta_3x_2^2+\varepsilon$, then the partial effect of $x_2$ is $\beta_2+2\beta_3x_2$ and depends on the level of $x_2$. This may happen in other models too, e.g. in linear spline models, but not in a simple linear (1st degree) model. Am I wrong? $\endgroup$
    – Sergio
    May 24, 2014 at 15:03
  • $\begingroup$ @rolando2: I am not sure whether we talk about the samte thing. If I include only the regular predictor variable I will get an estimated coefficient for that predictor that is either positive or negative. Based on the coefficient I can say that by adding one unit to x, y will increase or decrease by a certain amount. But I cannot find out this way whether small values actually lead to an increase of y, while higher values (from a certain unknown point on) lead to a decrease of y. $\endgroup$
    – Peter
    May 25, 2014 at 11:57
  • $\begingroup$ @Peter - I understand and I suggest you edit your question's "I assume" sentence to read: "I assume that, in some region of x, higher values of x have a positive effect on the dependent variable, while in some other region, higher values have a negative effect." $\endgroup$
    – rolando2
    May 26, 2014 at 22:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.