1
$\begingroup$

If I have two regressors that should theoretically have opposite signed coefficients, should I switch the sign (i.e. multiply by -1) of one of the variables before creating an interaction term?

In my work, I found the 'correct' signs for the main regressors, one positive and one negative in the non-interacted regression. When I created an interaction term between the two, the main effects kept their original signs but the interaction term was insignificant.

I switched the signs of one of the variables (multiplied each observation by -1) and re-ran the regressions. Obviously, without the interaction term, the absolute value of the coefficient on that variable remained the same but the sign had switched. However, when I added the new interaction term, the coefficient on the interaction term was now significant.

Note that these are all continuous variables and interaction terms were based on mean-centered variables as in Balli & Sorensen (2013).

I'm not sure why this would happen. Is this acceptable? Is it normal? Should I be concerned about my data?

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ What you describe cannot happen, because the significance of the regression results will be wholly unaffected by any constant rescaling of any of the variables. Because your procedure rescaled one of the regressors by $-1$ and (consequently) the interaction also by $-1$, it should not have changed any p-value whatsoever. $\endgroup$
    – whuber ♦
    Jul 27, 2016 at 19:46
  • $\begingroup$ I agree with whuber, algebraically it is not possible. Something else probably happened (like some observations ended up as missing, perhaps, or some other undiagnosed change). Without a reproducible example or a very careful accounting of what you did I doubt anyone will be able to confidently suggest what happened. $\endgroup$
    – Glen_b
    Jul 28, 2016 at 5:11
  • $\begingroup$ Thank you for your comments. You're correct that the procedure should have simply switched the sign of the interaction term as well. The problem is that I had mean-centered the interaction term, meaning that some observations which had previously been positive were now negative (or vice versa). E.g. if the mean was 5, an observation of 3 would now become -2, creating an interaction term that is no longer simply the negative of the original interaction term. What I find odd is that it is this (incorrect) interaction term that is significant in the model. Spurious I suppose. $\endgroup$
    – Simon
    Jul 28, 2016 at 21:25
  • $\begingroup$ The shifting that @Simon inadvertently caused is the key for why the significance of his fit changed. Scaling by itself will have no effect. See my answer for more, this really messed me up for a while. $\endgroup$
    – Nick Crews
    Mar 25, 2022 at 18:53

2 Answers 2

Reset to default
1
$\begingroup$

Whenever you add a new variable in a regression analysis, all coefficients may change. They will always change unless the added variable is uncorrelated to all the previous variable. The extreme case where two or more variables are highly correlated is multicollinearity.

And about your question of switching signs of one of the variables, it will just change the signs of some of your coefficients. In fact, applying any linear transformation to any variable won't change anything; coefficients will just get transformed in the same way to keep yielding the same predictions for the same situation.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Your statement that applying any linear transformation to any variable won't change anything is wrong here. It's only correct for regressions without interaction terms. With interaction terms, linear transformations in the source variables can cause non-linear transformations in the interaction variable. See my answer. $\endgroup$
    – Nick Crews
    Mar 25, 2022 at 18:49
1
$\begingroup$

Resurrecting this, because previous answers discuss changing a variable by a scaling factor (eg multiplying by -1), but I think it's important to talk about the general case, changing a variable by a linear transformation (spoiler, this case changes your fit!).

If you adjust one of your independent variables by simply a scaling factor (eg x' = 2x), then as the answers above say, the fit of the model doesn't change, some coefficients will just adjust themselves. This is because the interaction term is changed by merely a scaling, which is a linear transformation. Say originally you had x and y, and their interaction i = xy. If now you say that x' = 2x, then your interaction variable is now i' = x'y = 2xy = 2i. i undergoes a linear transformation, because i' is simply twice the original i.

However, if you adjust one of your independent variables by shifting (eg x' = x + 1), then all of a sudden the interaction term can change nonlinearly, and the fit of your model can change! For example, say x' = x + 1, then your interaction variable is now i' = x'y = (x+1)y = xy + y. This is NOT a linear transformation of i, because you can't express i' in the form i' = i*A + B (where A and B are constants).

Note that shifting, just like scaling, is still an example of a linear transformation, so it's non-obvious that these two cases are going to act differently. On the surface this seems to contradict an idea you may have learned in the past, that "applying any linear transformation to a variable won't change anything". And that idea is true when dealing with linear regression without interaction terms. It's the interaction term that makes this idea no longer hold.

Places to watch out for this:

  • Categorical or boolean variables, e.g. 0 or 1. If you just swap them, then you have a shift.
  • Another example is in the OP's follow up comment where mean-centering inadvertently added a shift (-3 turned into 2).
  • Others?
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.