2
$\begingroup$

$y=\beta_1+\beta_2x_2+\beta_3x_3+\beta_4x_3^2+\beta_5x_5+\beta_6x_6$ for a sample size of 50.

Doing an F-test for joint significance, t-tests gives that $x_3,x_3^2,x_5$ are not significant. If I remove $x_3^2,x_5$, then $x_3$ becomes significant. Why?

At first I thought of some collinearity, but one is the square of the other. Maybe I have too many regressors for this sample size?

Any help would be appreciated.

Improve this question
$\endgroup$
3
  • 3
    $\begingroup$ Unless $x_3$ has mean 0, then $x_3$ and $x_3^2$ are likely to be correlated, potentially very strongly. $\endgroup$
    – Cliff AB
    Commented Mar 22, 2017 at 16:20
  • $\begingroup$ @CliffAB Thanks for your comment. In my case, $x_3$ takes only positive integers, hence cannot have zero mean. But out of curiosity, if that were the case, how would you justify it? If they are very correlated, why would this happen? $\endgroup$
    – An old man in the sea.
    Commented Mar 22, 2017 at 16:23
  • 2
    $\begingroup$ So it sounds like $x_3$ and $x_3^2$ are likely to be at least moderately correlated, perhaps strongly. A heuristic way to see why this can create problems is that if $x_1$ and $x_2$ are strongly correlated, you would have a lot of trouble accurately separating the effect of $x_1$ from the effect of $x_2$. Once you take one of the variables out of the model, you no longer have to separate the effects. $\endgroup$
    – Cliff AB
    Commented Mar 22, 2017 at 16:30

1 Answer 1

Reset to default
2
$\begingroup$

When you include $x_3^2$, the test of $x_3$ becomes a test of $x_3$ when $x_3=0$ even if that value is not possible. Since that is unlikely to be of interest to you, you could center $x_3$ even though it only takes on integer values. The effect of $x_3$ then becomes a test of the mean effect of $x_3$. Many people focus on collinearity in this situation. Although technically correct, the main thing is that including $x_3^2$ changes the meaning of the effect of $x_3$. If there is no evidence of a quadratic component, the best course of action is probably to leave out $x_3^2$.

Improve this answer
$\endgroup$
2
  • $\begingroup$ David Lane, thanks for your answer. Could you please elaborate a bit more? «The effect of x3 then becomes a test of the mean effect of x3» Not sure how to interpret this. sorry for my lack of knowledge. ;) $\endgroup$
    – An old man in the sea.
    Commented Mar 22, 2017 at 18:59
  • $\begingroup$ If there is a quadratic component that means the effect of x3 changes as a function of the level of x3. The test of x3 is essentially a simple effect in the sense that it is the effect at a single level of x3. It turns out that it is the effect of x3 when x3=0. If you make the mean of x3 0, then the effect of x3 is tested at the mean level of x3. $\endgroup$
    – David Lane
    Commented Mar 22, 2017 at 19:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.