7
$\begingroup$

I am currently running a multiple linear regression, and I am bit confused in regards to how to properly add interaction terms to the model by hand. All of the variables I am using are continuous and have different scales and units.

So far, the way I have done it has been to

  1. Standardize the observations for each variables

  2. Multiply corresponding standardized values from specific variables to create the interaction terms and then add these new variables to the set of regression data

  3. Run the regression

Is this the correct way to go about doing this? Should I standardize the interaction term variables also after calculating the 'raw' terms?

Improve this question
$\endgroup$

3 Answers 3

Reset to default
6
$\begingroup$

The approach in the question seems to be correct as long as the variables of concern are continuous or binary. Categorical variables with three or more levels cannot be multiplied as stated.

The standardized interaction term should be the standardized version of the product of the two original variables, not the product of the two standardized variables. Here is an example using the sample data set auto in Stata:

Let's say we are interested in using mile per gallon (mpg), weight of the car (weight) and their interaction to predict the price (price). The original model is:

. reg price mpg weight c.mpg#c.weight

      Source |       SS       df       MS              Number of obs =      74
-------------+------------------------------           F(  3,    70) =   13.11
       Model |   228430463     3  76143487.7           Prob > F      =  0.0000
    Residual |   406634933    70  5809070.47           R-squared     =  0.3597
-------------+------------------------------           Adj R-squared =  0.3323
       Total |   635065396    73  8699525.97           Root MSE      =  2410.2

--------------------------------------------------------------------------------
         price |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
---------------+----------------------------------------------------------------
           mpg |   396.7844   185.2023     2.14   0.036     27.41003    766.1587
        weight |   5.067008   1.378057     3.68   0.000      2.31856    7.815455
               |
c.mpg#c.weight |  -.1916795   .0711936    -2.69   0.009    -.3336706   -.0496885
               |
         _cons |  -5944.881   4525.706    -1.31   0.193    -14971.12    3081.356
--------------------------------------------------------------------------------

If we standardized the product, the results will agree with the original:

. reg price zmpg zwt zmpgWeight

      Source |       SS       df       MS              Number of obs =      74
-------------+------------------------------           F(  3,    70) =   13.11
       Model |   228430457     3  76143485.6           Prob > F      =  0.0000
    Residual |   406634939    70  5809070.56           R-squared     =  0.3597
-------------+------------------------------           Adj R-squared =  0.3323
       Total |   635065396    73  8699525.97           Root MSE      =  2410.2

------------------------------------------------------------------------------
       price |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        zmpg |   2295.597   1071.489     2.14   0.036     158.5807    4432.614
         zwt |   3938.046   1071.017     3.68   0.000      1801.97    6074.121
  zmpgWeight |  -1773.852   658.8436    -2.69   0.009    -3087.874   -459.8299
       _cons |   6165.257   280.1802    22.00   0.000     5606.455    6724.059
------------------------------------------------------------------------------

However, if we use the product of the standardized variables, the results will different than the original. ANOVA results are the same, but you can see the p-values of the standardized mpg and weight are different:

. reg price zmpg zwt c.zmpg#c.zwt

      Source |       SS       df       MS              Number of obs =      74
-------------+------------------------------           F(  3,    70) =   13.11
       Model |   228430459     3  76143486.3           Prob > F      =  0.0000
    Residual |   406634937    70  5809070.53           R-squared     =  0.3597
-------------+------------------------------           Adj R-squared =  0.3323
       Total |   635065396    73  8699525.97           Root MSE      =  2410.2

------------------------------------------------------------------------------
       price |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        zmpg |   -1052.87   556.2308    -1.89   0.063    -2162.238    56.49692
         zwt |   765.3424    526.041     1.45   0.150    -283.8133    1814.498
             |
c.zmpg#c.zwt |  -861.8786   320.1187    -2.69   0.009    -1500.335    -223.422
             |
       _cons |   5478.971   378.7809    14.46   0.000     4723.517    6234.426
------------------------------------------------------------------------------
Improve this answer
$\endgroup$
7
  • $\begingroup$ Thanks for the response. So, if I follow your advice correctly, you're saying that I should first find the product of raw variables and then standardize the new interaction variable for use in the regression? Should I center the two variables before finding their product, or should I simply use the raw values? $\endgroup$
    – dwm8
    May 8, 2015 at 19:42
  • $\begingroup$ I think le_andrew has given a great account on the two examples I showed. Both ways give the same model, as indicated by the identical ANOVA results, but the interpretation of mpg and weight variable will differ. Yet, I am more of a fan of the approach suggested by Aksakal, I do not like standardization because their interpretation (in z-score) is unintuitive. $\endgroup$
    – Penguin_Knight
    May 8, 2015 at 20:24
  • $\begingroup$ Correct me if I'm wrong, but are you basically saying that it is wrong to calculate an interaction term by multiplying standardized or centered values? $\endgroup$
    – dwm8
    May 8, 2015 at 20:41
  • $\begingroup$ @milesda, of course I don't mean it's wrong, people have been doing this for a long time as a way to make the coefficients comparable. My problem with this approach is that the interpretation becomes "per one standard deviation change," and in my own field (biomedical) most predictors do not change in the unit of SD. I prefer keeping their original units. $\endgroup$
    – Penguin_Knight
    May 8, 2015 at 20:59
  • $\begingroup$ Thanks for the help, and sorry for the dumb questions, I'm pretty new at this. The idea that the first two examples yield identical results makes sense to me, as well as the fact that the third is different. That being said, my main question is: knowing that the two methods yield different results, which of the two options makes the most sense mathematically, and why? I guess I'm having a hard time wrapping my head around the implications of doing it one way versus the other. $\endgroup$
    – dwm8
    May 8, 2015 at 21:26
5
$\begingroup$

  1. Standardization is not a requirement, but is an option. Mean-centering (a part of standardization) makes the lower order terms more interpretable. Penguin_Knight showed that standarizing after forming the interaction term rather than before gives you the same results as the unstandardized model. Note that this is a consequence of the change in interpretation of lower order terms when you mean-center variables before forming the interaction term. Both of his outputs are valid (note the interaction t value is identical) you just need to know how to interpret the lower order coefficients (the main effects in ANOVA terms). In short, when you mean-center/standardize before forming your interaction terms, the mpg effect is the effect of mpg when for an average weight car (because it is the effect when all other variables it interacts with is 0, and for the weight variables we set 0 to equal the mean). Without mean-centering/standardizing, the mpg effect is the effect of mpg for a car that weighs 0 pounds (hence mean-centering usually improves interpretability since cars can't weight 0 points).

  2. Is correct but missing some details. For continuous variables, you only need to multiply two variables to form an interaction (again after mean-centering or standardizing if you wish). When categorical variables are involved, you can create an interaction term by first creating separate numerical variables that correspond to contrasts of interest. You can create as many contrasts as you have levels of your categorical variable minus 1. You do not need to use a full set of contrast codes, however. Once you have your columns of contrast codes, you create your interactions the same as before as you now are merely multiplying two numerical variables. Note: this works for interactions of categorical:categorical and categorical:continuous and any permutation at higher orders of interactions.

  3. Run your regression. I have been assuming that you also have the lower order variables in the model as well (i.e. $y=a+b+ab$ rather than $y=ab$ which would adjust how you interpret the results).

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for the response. I should've mentioned that all of my IVs are continuous, most of which have different scales and units. Just to clarify, it sounds like what you're suggesting is a bit different than Penguin_Knight. Knowing that all of the variables are continuous, would you suggest standardizing all variables, calculating interaction terms using standardized values, and then running the regression without standardizing the interaction terms? $\endgroup$
    – dwm8
    May 8, 2015 at 20:21
  • 1
    $\begingroup$ It all depends on what you're trying to do. Having variables in different units and scales doesn't matter in an objective sense. The regression is the same if you rescale your predictors. What changes is what the coefficients mean. Are you interpreting the coefficients? Do you care about simple effects (i.e the effect of one variable at different levels of the other variable it interacts with)? There is no suggestion to be made without knowing what you are trying to do. $\endgroup$
    – le_andrew
    May 9, 2015 at 20:49
1
$\begingroup$

You don't need to standardize anything unless the scales are vastly different. Even in this case you don't necessarily need to standardize, a simple unit of measure (scale) will work fine. Once you changes the scale, then the interactions would be on scaled variables too.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.