25
$\begingroup$

I start with my OLS regression: $$ y = \beta _0 + \beta_1x_1+\beta_2 D + \varepsilon $$ where D is a dummy variable, the estimates become different from zero with a low p-value. I then preform a Ramsey RESET test and find that i have some misspesification of the equation, i thus include squared x: $$ y = \beta _0 + \beta_1x_1+\beta_2x_1^2+\beta_3 D + \varepsilon $$

  1. What does the squared term explain? (Non-linear increase in Y?)
  2. By doing this my D estimate does not vary from zero any more, with a high p-value. How do i interpret the squared term in my equation (in general)?
$\endgroup$
11
  • $\begingroup$ possible duplicate of Why ANOVA/Regression results change when controlling for another variable $\endgroup$
    – Macro
    Commented Mar 18, 2013 at 13:15
  • 1
    $\begingroup$ Probable reason: $x_{1}^2$ and $D$ seem to explain the same variablility in $y$ $\endgroup$
    – steadyfish
    Commented Mar 18, 2013 at 13:17
  • 3
    $\begingroup$ One thing that might help is to center $x$ before creating your squared term (see here). As for the interpretation of your squared term, I argue that it's best to interpret $\beta_1x_1+\beta_2x_1^2$ as a whole (see here). Another thing is that you may need an interaction, that means adding $\beta_4x_1D+\beta_5x_1^2D$. $\endgroup$ Commented Mar 18, 2013 at 13:45
  • $\begingroup$ I don't think it's really a duplicate of that question; the solution is different (centering variables works here, but not there, unless I am mistaken) $\endgroup$
    – Peter Flom
    Commented Mar 18, 2013 at 14:13
  • $\begingroup$ @Peter, I interpret this question as a subset of "Why is it that when I add a variable to my model, the effect estimate/$p$-value for some other variable changes?", which is addressed in the other question. Among the answers to that questions are collinearity (which gung does allude to in his answer to that question)/content overlap between predictors (i.e. between $D$ and $(x_1,x_1^2)$, which I suspect is the culprit in this case). The same logic applies here. I'm not sure what the controversy is but that's fine if you and others disagree. Cheers. $\endgroup$
    – Macro
    Commented Mar 18, 2013 at 14:27

2 Answers 2

25
$\begingroup$

Well, first of, the dummy variable is interpreted as a change in intercept. That is, your coefficient $\beta_3$ gives you the difference in the intercept when $D=1$, i.e. when $D=1$, the intercept is $\beta_0 + \beta_3$. That interpretation doesn't change when adding the squared $x_1$.

Now, the point of adding a squared to the series is that you assume that the relationship wears off at a certain point. Looking at your second equation

$$y = \beta _0 + \beta_1x_1+\beta_2x_1^2+\beta_3 D + \varepsilon$$

Taking the derivate w.r.t. $x_1$ yields

$$\frac{\delta y}{\delta x_1} = \beta_1 + 2\beta_2 x_1$$

Solving this equation gives you the turning point of the relationship. As user1493368 explained, this is indeed reflecting an inverse U-shape if $\beta_1<0$ and vice versa. Take the following example:

$$\hat{y} = 1.3 + 0.42 x_1 - 0.32 x_1^2 + 0.14D$$

The derivative w.r.t. $x_1$ is

$$\frac{\delta y}{\delta x_1} = 0.42 - 2\times 0.32 x_1 $$

Solving for $x_1$ gives you

$$\frac{\delta y}{\delta x_1} = 0 \iff x_1 \approx 0.66 $$

That is the point at which the relationship has its turning point. You can take a look at Wolfram-Alpha's output for the above function, for some visualization of your problem.

Remember, when interpreting the ceteris paribus effect of a change in $x_1$ on $y$, you have to look at the equation:

$$\Delta y = (\beta_1 + 2\beta_2x_1)\Delta x$$

That is, you can not interpret $\beta_1$ in isolation, once you added the squared regressor $x_1^2$!

Regarding your insignificant $D$ after including the squared $x_1$, it points towards misspecification bias.

$\endgroup$
5
  • $\begingroup$ Hi. If you had several predictors should you use partial derivatives or total derivatives (diferentials)? $\endgroup$
    – skan
    Commented Aug 24, 2016 at 0:06
  • 1
    $\begingroup$ A partial derivative is still the right way to go here. The interpretation of all coefficients is ceteris paribus, i.e., holding everything else constant. That's exactly what you are doing when you take a partial derivative. $\endgroup$
    – altabq
    Commented Aug 26, 2016 at 9:07
  • $\begingroup$ See this UCLA IDRE page to complement @altabq's great answer. $\endgroup$
    – Cyrille
    Commented Oct 18, 2018 at 7:22
  • $\begingroup$ @altabq Even though beta 1 is negative, aren't there cases where it isn't an inverse U-shape? $\endgroup$ Commented Apr 4, 2023 at 4:57
  • $\begingroup$ @chunguc1004, you are right. The shape of the curve depends on beta 1. If beta 1 is positive, we have a U-shape and vice versa for negative beta 1. $\endgroup$
    – altabq
    Commented Jul 18, 2023 at 12:03
24
$\begingroup$

A good example of including square of variable comes from labor economics. If you assume y as wage (or log of wage) and x as an age, then including x^2 means that you are testing the quadratic relationship between an age and wage earning. Wage increases with the age as people become more experienced but at the higher age, wage starts to increase at decreasing rate (people becomes older and they will not be so healthy to work as before) and at some point the wage doesn't grow (reaches the optimal wage level) and then starts to fall (they retire and their earnings starts to decrease). So, the relationship between wage and age is inverted U-shaped (life cycle effect). In general, for the example mentioned here, the coefficient on age is expected to be positive and than on age^2 to be negative.The point here is that there should be theoretical basis /empirical justification for including the square of the variable. The dummy variable, here, can be thought of as representing gender of the worker. You can also include interaction term of gender and age to examine the whether the gender differential varies by age.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.