# Linear Regression - I remove 2 of 3 non-significant regressors, and the remaining one becomes significant. Why?

$y=\beta_1+\beta_2x_2+\beta_3x_3+\beta_4x_3^2+\beta_5x_5+\beta_6x_6$ for a sample size of 50.

Doing an F-test for joint significance, t-tests gives that $x_3,x_3^2,x_5$ are not significant. If I remove $x_3^2,x_5$, then $x_3$ becomes significant. Why?

At first I thought of some collinearity, but one is the square of the other. Maybe I have too many regressors for this sample size?

Any help would be appreciated.

• Unless $x_3$ has mean 0, then $x_3$ and $x_3^2$ are likely to be correlated, potentially very strongly. – Cliff AB Mar 22 '17 at 16:20
• @CliffAB Thanks for your comment. In my case, $x_3$ takes only positive integers, hence cannot have zero mean. But out of curiosity, if that were the case, how would you justify it? If they are very correlated, why would this happen? – An old man in the sea. Mar 22 '17 at 16:23
• So it sounds like $x_3$ and $x_3^2$ are likely to be at least moderately correlated, perhaps strongly. A heuristic way to see why this can create problems is that if $x_1$ and $x_2$ are strongly correlated, you would have a lot of trouble accurately separating the effect of $x_1$ from the effect of $x_2$. Once you take one of the variables out of the model, you no longer have to separate the effects. – Cliff AB Mar 22 '17 at 16:30

When you include $x_3^2$, the test of $x_3$ becomes a test of $x_3$ when $x_3=0$ even if that value is not possible. Since that is unlikely to be of interest to you, you could center $x_3$ even though it only takes on integer values. The effect of $x_3$ then becomes a test of the mean effect of $x_3$. Many people focus on collinearity in this situation. Although technically correct, the main thing is that including $x_3^2$ changes the meaning of the effect of $x_3$. If there is no evidence of a quadratic component, the best course of action is probably to leave out $x_3^2$.