Why could centering independent variables change the main effects with moderation?

Question

I have a question related to multiple regression and interaction, inspired by this CV thread: Interaction term using centered variables hierarchical regression analysis? What variables should we center?

When checking for a moderation effect I do center my independent variables and multiply the centered variables in order to calculate my interaction term. Then I run my regression analysis and check for main and interaction effects, which may show the moderation.

If I redo the analysis without centering, apparently the coefficient of determination ($R^2$) does not change but the regression coefficients ($\beta$s) do. That seems clear and logical.

What I do not understand: The p-values of the main effects change substantially with centering, although the interaction does not (which is right). So my interpretation of main effects could change dramatically - just determined by centering or not. (It's is still the same data, in both analyses!)

Can somebody clarify? - Because that would mean that the option to center my variables would be mandatory and everybody should do it in order to get the same results with the same data.

---

Thanks a lot for distributing to that problem and your comprehensive explanations. Be assured that your help is very much appreciated!

For me, the biggest advantage of centering is to avoid multicollinearity. It is still quite confusing to establish a rule, whether to center or not. My impression is, that most of the resources suggest to center, although there are some "risks" when doing it. Again I want to put out the fact, that 2 researchers dealing with the same material and data might conclude different results, because one does centering and the other does not. I just read some part of a book by Bortz (he was a Professor and kind of a Statistics Star in Germany and Europe), and he does not even mention that technique; just points out to be careful in interpreting main effects of variables when they are involved in interactions.

After all, when you conduct a regression with one IV, one moderator (or second IV) and a DV, would you recommend to center or not?

Answer 1

In models with no interaction terms (that is, with no terms that are constructed as the product of other terms), each variable's regression coefficient is the slope of the regression surface in the direction of that variable. It is constant, regardless of the values of the variables, and therefore can be said to measure the overall effect of that variable.

In models with interactions, this interpretation can be made without further qualification only for those variables that are not involved in any interactions. For a variable that is involved in interactions, the "main-effect" regression coefficient -- that is, the regression coefficient of the variable by itself -- is the slope of the regression surface in the direction of that variable when all other variables that interact with that variable have values of zero, and the significance test of the coefficient refers to the slope of the regression surface only in that region of the predictor space. Since there is no requirement that there actually be data in that region of the space, the main-effect coefficient may bear little resemblance to the slope of the regression surface in the region of the predictor space where data were actually observed.

In anova terms, the main-effect coefficient is analogous to a simple main effect, not an overall main effect. Moreover, it may refer to what in an anova design would be empty cells in which the data were supplied by extrapolating from cells with data.

For a measure of the overall effect of the variable that is analogous to an overall main effect in anova and does not extrapolate beyond the region in which data were observed, we must look at the average slope of the regression surface in the direction of the variable, where the averaging is over the N cases that were actually observed. This average slope can be expressed as a weighted sum of the regression coefficients of all the terms in the model that involve the variable in question.

The weights are awkward to describe but easy to get. A variable's main-effect coefficient always gets a weight of 1. For each other coefficient of a term involving that variable, the weight is the mean of the product of the other variables in that term. For example, if we have five "raw" variables x1, x2, x3, x4, x5, plus four two-way interactions (x1,x2), (x1,x3), (x2,x3), (x4,x5), and one three-way interaction (x1,x2,x3), then the model is

y = b0 + b1*x1 + b2*x2 + b3*x3 + b4*x4 + b5*x5 +
    b12*x1*x2 + b13*x1*x3 + b23*x2*x3 + b45*x4*x5 +
    b123*x1*x2*x3 + e

and the overall main effects are

B1 = b1 + b12*M[x2] + b13*M[x3] + b123*M[x2*x3],

B2 = b2 + b12*M[x1] + b23*M[x3] + b123*M[x1*x3],

B3 = b3 + b13*M[x1] + b23*M[x2] + b123*M[x1*x2],

B4 = b4 + b45*M[x5],

B5 = b5 + b45*M[x4],

where M[.] denotes the sample mean of the quantity inside the brackets. All the product terms inside the brackets are among those that were constructed in order to do the regression, so a regression program should already know about them and should be able to print their means on request.

In models that have only main effects and two-way interactions, there is a simpler way to get the overall effects: center[1] the raw variables at their means. This is to be done prior to computing the product terms, and is not to be done to the products. Then all the M[.] expressions will become 0, and the regression coefficients will be interpretable as overall effects. The values of the b's will change; the values of the B's will not. Only the variables that are involved in interactions need to be centered, but there is usually no harm in centering other measured variables. The general effect of centering a variable is that, in addition to changing the intercept, it changes only the coefficients of other variables that interact with the centered variable. In particular, it does not change the coefficients of any terms that involve the centered variable. In the example given above, centering x1 would change b0, b2, b3, and b23.

[1 -- "Centering" is used by different people in ways that differ just enough to cause confusion. As used here, "centering a variable at #" means subtracting # from all the scores on the variable, converting the original scores to deviations from #.]

So why not always center at the means, routinely? Three reasons. First, the main-effect coefficients of the uncentered variables may themselves be of interest. Centering in such cases would be counter-productive, since it changes the main-effect coefficients of other variables.

Second, centering will make all the M[.] expressions 0, and thus convert simple effects to overall effects, only in models with no three-way or higher interactions. If the model contains such interactions then the b -> B computations must still be done, even if all the variables are centered at their means.

Third, centering at a value such as the mean, that is defined by the distribution of the predictors as opposed to being chosen rationally, means that all coefficients that are affected by centering will be specific to your particular sample. If you center at the mean then someone attempting to replicate your study must center at your mean, not their own mean, if they want to get the same coefficients that you got. The solution to this problem is to center each variable at a rationally chosen central value of that variable that depends on the meaning of the scores and does not depend on the distribution of the scores. However, the b -> B computations still remain necessary.

The significance of the overall effects may be tested by the usual procedures for testing linear combinations of regression coefficients. However, the results must be interpreted with care because the overall effects are not structural parameters but are design-dependent. The structural parameters -- the regression coefficients (uncentered, or with rational centering) and the error variance -- may be expected to remain invariant under changes in the distribution of the predictors, but the overall effects will generally change. The overall effects are specific to the particular sample and should not be expected to carry over to other samples with different distributions on the predictors. If an overall effect is significant in one study and not in another, it may reflect nothing more than a difference in the distribution of the predictors. In particular, it should not be taken as evidence that the relation of the dependent variable to the predictors is different in the two studies.

Answer 2

That is because in any regression involving more than one predictor, the $\beta$s are partial coefficients; they are interpreted as the predicted change in the dependent variable for each 1-unit increase in a predictor, holding all other predictors constant.

In a regression involving interaction terms, for example $y=\beta_1x_1+\beta_2x_2+\beta_3x_1x_2+\epsilon$, $\beta_1$ is the expected increase in the dependent variable for each 1-unit increase in $x_1$, holding all the other terms constant. This is a problem for the term $\beta_3x_1x_2$, as it will vary as $x_1$ varies. The only way to hold the interaction term constant for a 1-unit increase on either $x_1$ or $x_2$ (the two variables involved in the interaction) is to set the other variable to 0. Therefore, when a variable is also part of an interaction term, the interpretation of the $\beta$ for this variable is conditional on the other variable being 0—not merely being held constant.

For this reason, the interpretation of the $\beta$s will change depending on where the 0 is on the other variable involved in the interaction; where the 0 is on the variable of interest does not actually change the interpretation of its coefficient. In this case, for example, $\beta_1$ is the predicted increase in $y$ for each 1-unit increase in $x_1$ when $x_2=0$. If the relationship between $x_1$ and $y$ changes as a function of $x_2$ (as you hypothesize it does when you include an interaction term), then the significance of $\beta_1$ will change as a function of the centering of $x_2$.

Also, note that if the value of your $\beta$s change considerably as a function of centering, then your interaction term is probably significant; and if it is, interpreting the "main effects" can be misleading, because this means that the relationship between $x_1$ and $y$ depends on the value of $x_2$, and vice versa. A typical way to deal with this is to plot predicted values for $y$ as a function of $x_1$, for a few values of $x_2$ (say, 3; for example, 0 and ±1 SD).

Answer 3

I have been going crazy with the same question, but i finally found the solution to your and my problem. IT IS ALL ABOUT HOW YOU CALCULATE YOUR CENTERED VARIABLES. Two options are available:
1. MEAN - INDIVIDUAL VARIABLES 2. INDIVIDUAL VARIABLES - MEAN
You probably calculated your centered variables as (individual variable - mean value), therefore those with low values would get negative scores, and those with high values would get positive scores.
I’ll explain with an example to make it easier to understand. I want to see how muscle strength, affects bone mass and I want to take into account gender to see if it affects differently in girls and boys. The idea is that the higher the muscle strength the higher the bone mass. I therefore have:

Dependent variable: Bone mass Independent variables: Sex, muscle strength, interaction_SEX_MUSCLEstrength.

As I found multicollinearity (you usually do when you have an interaction term), I centred musclestrength (MEAN – INDIVIDUAL VARIABLE) and created the new interaction term with the new centred variable. My coefficients were

Constant: 0.902
Gender: -0.010 (Boys = 0; Girls =1)
Centred muscle: -0.023
Interaction: 0.0002
Therefore if you wanted to estimate a boys bone mass you would have the following equation:
Bone mass = $0.902 - (0 * 0.010) – (0.023 * muscle centred value) + (Interaction * 0.0002)$

Looking at this you might think that muscle is affecting bone negatively, but you have to think of your centred variables, not your original variables. Let’s say the mean muscle strength of the group was of 30 KG. And you want to estimate the bone mass of a boy (WEAKBOY) that performed 20 KG and another that performed 40KG (STRONGBOY). The centred values of WEAKBOY will be (MEAN GROUP VALUE – INDIVIDUAL VALUE; 30 – 20 = 10), and for STRONGBOY will be -10. Applying these values to the equation:

WEAKBOY Bone mass= 0.902 – 0 – (0.023*10) + ....=0.672

STRONGBOY Bone mass= 0.902 – (0.023*(-10)) + ...= 1.132

As you can see STRONGBOY will indeed had a stronger bone. If you had centred your variables the other way round: (INDIVIDUAL – MEAN), all the coefficients will be the same but the symbols will be different. This is because when you apply the centred variable WEAKBOY will be (-10) and STRONGBOY will be (+10). Therefore the final results will be exactly the same.

It all makes sense once you understand it.

Hope the example is clear enough.