17
$\begingroup$

I have two main effects, V1 and V2. The effects of V1 and V2 on the response variables are negative. However, for some reason I am getting positive coefficient for the interaction term V1*V2. How can I interpret this? is such situation possible?

$\endgroup$
  • 3
    $\begingroup$ Absolutely. It can be interpreted as a reduction of V1's inverse estimated effect across levels of V2 (or vice versa), i.e. the inverse effect of V1 isn't as inverse for higher observations of V2. You should plot everything to verify. $\endgroup$ – D L Dahly Dec 18 '13 at 18:27
  • $\begingroup$ The main effect coefficients are the slope of the response surface in the V1 and V2 directions at the point V1 = V2 = 0. If your model contains an intercept then try centering V1 and V2 (i.e., subtract their means). The interaction is the product of the centered V1 and V2; it is not separately centered, and its coefficient should not change. $\endgroup$ – Ray Koopman Dec 18 '13 at 18:44
  • $\begingroup$ I believe yours is a slightly different issue, but you may find Simpson's Paradox interesting: en.wikipedia.org/wiki/Simpson's_paradox $\endgroup$ – David Marx Dec 18 '13 at 21:56
28
$\begingroup$

Certainly. As a simple example, consider an experiment where you are adding certain volumes of hot (V1) and cold (V2) water to a fish tank that begins at the correct temperature. The response variable (V3) is the number of fish that survive after a day. Intuitively, if you add only hot water (V1 increases), lots of fish will die (V3 goes down). If you add only cold water (V2 increases), lots of fish will die (V3 goes down). But if you add both hot and cold water (V1 and V2 increases, thus V1*V2 increases), the fish will be fine (V3 stays high), so the interaction must counteract the two main effects and be positive.

Below, I made up 18 data points mimicking the above situation and fit multiple linear regression in R and included the output. You can see the two negative main effects and positive interaction in the last line. You can let V1 = Liters of hot water, V2 = Liters of cold water, and V3 = Number of fish alive after one day.

   V1 V2  V3
1   0  0 100
2   0  1  90
3   1  0  89
4   1  1  99
5   2  0  79
6   0  2  80
7   2  1  91
8   1  2  92
9   2  2  99
10  3  3 100
11  2  3  88
12  3  2  91
13  0  3  70
14  3  0  69
15  3  3 100
16  4  0  61
17  0  4  60
18  4  2  82

A = matrix(c(0,0,100, 0,1,90, 1,0,89, 1,1,99, 2,0,79, 0,2,80, 2,1,91, 1,2,92, 
2,2,99, 3,3,100, 2,3,88, 3,2,91, 0,3,70, 3,0,69, 3,3,100, 4,0,61, 0,4,60, 
4,2, 82), byrow=T, ncol=3)

A = as.data.frame(A)

summary(lm(V3~V1+V2+V1:V2, data=A))


Coefficients:
(Intercept)           V1           V2        V1:V2  
    103.568      -10.853      -10.214        6.563  
$\endgroup$
  • 8
    $\begingroup$ Clever example. $\endgroup$ – D L Dahly Dec 19 '13 at 13:30
5
$\begingroup$

An alternative way of looking at the situation to @underminer's brilliant example is to note that under least squares regression, your fitted values satisfy "correlation constraints"

$$\sum_{ i=1}^nx_{ik}\hat{y}_i=\sum_{ i=1}^nx_{ik}y_i$$

Where $ x_{ik} $ is the value of the kth (independent/explanatory/predictor/etc) variable on the ith observation. Note that the right hand side does not depend on what other variables are in the model. So if "y" generally increases/decreases with the kth variable then the fitted values also will. This is easy to see through the betas when only main effects are present, but confusing when interactions are present.

Note how interactions generally "ruin" the typical interpretation of betas as "effect on the response by increasing that variable by one unit with all other variables held constant ". This is a useless interpretation when interactions are present as we know that varying a single variable will alter the values for the interaction terms as well as the main effects. In the most simple case given by your example you have that changing $V1$ by one will alter the fitted value by

$$\beta_1 + V2\beta_{1*2} $$

Clearly just looking at $\beta_1 $ won't give you the proper "effect" of $ V1 $ on the response.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.