25
$\begingroup$

I have data from a survey experiment in which respondents were randomly assigned to one of four groups:

> summary(df$Group)
       Control     Treatment1     Treatment2     Treatment3 
            59             63             62             66 

While the three treatment groups do vary slightly in the stimulus applied, the main distinction that I care about is between the control and treatment groups. So I defined a dummy variable Control:

> summary(df$Control)
     TRUE FALSE 
       59   191 

In the survey, respondents were asked (among other things) to choose which of two things they preferred:

> summary(df$Prefer)
      A   B  NA's 
    152  93   5 

Then, after receiving some stimulus as determined by their treatment group (and none if they were in the control group), respondents were asked to choose between the same two things:

> summary(df$Choice)
  A    B 
149  101 

I want to know if the being in one of the three treatment groups had an effect on the choice that respondents made in this last question. My hypothesis is that respondents who received a treatment are more likely to choose A than B.

Given that I am working with categorical data, I have decided to use a logit regression (feel free to chime in if you think that's incorrect). Since respondents were randomly assigned, I am under the impression that I shouldn't necessarily need to control for other variables (e.g. demographics), so I have left those out for this question. My first model was simply the following:

> x0 <- glm(Product ~ Control + Prefer, data=df, family=binomial(link="logit"))
> summary(x0)

Call:
glm(formula = Choice ~ Control + Prefer, family = binomial(link = "logit"), 
    data = df)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.8366  -0.5850  -0.5850   0.7663   1.9235  

Coefficients:
                    Estimate Std. Error z value Pr(>|z|)    
(Intercept)           1.4819     0.3829   3.871 0.000109 ***
ControlFALSE         -0.4068     0.3760  -1.082 0.279224    
PreferA              -2.7538     0.3269  -8.424  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 328.95  on 244  degrees of freedom
Residual deviance: 239.69  on 242  degrees of freedom
  (5 observations deleted due to missingness)
AIC: 245.69

Number of Fisher Scoring iterations: 4

I am under the impression that the intercept being statistically significant is not something that holds interpretable meaning. I thought perhaps that I should include an interaction term as follows:

> x1 <- glm(Choice ~ Control + Prefer + Control:Prefer, data=df, family=binomial(link="logit"))
> summary(x1)

Call:
glm(formula = Product ~ Control + Prefer + Control:Prefer, family = binomial(link = "logit"), 
    data = df)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-2.5211  -0.6424  -0.5003   0.8519   2.0688  

Coefficients:
                                 Estimate Std. Error z value Pr(>|z|)    
(Intercept)                         3.135      1.021   3.070  0.00214 ** 
ControlFALSE                       -2.309      1.054  -2.190  0.02853 *  
PreferA                            -5.150      1.152  -4.472 7.75e-06 ***
ControlFALSE:PreferA                2.850      1.204   2.367  0.01795 *  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 328.95  on 244  degrees of freedom
Residual deviance: 231.27  on 241  degrees of freedom
  (5 observations deleted due to missingness)
AIC: 239.27

Number of Fisher Scoring iterations: 5

Now the respondents status as in a treatment group has the expected effect. Was this a valid set of steps? How can I interpret the interaction term ControlFALSE:PreferA? Are the other coefficients still the log odds?

$\endgroup$
31
$\begingroup$

I assume that PreferA = 1 when one prefered A and 0 otherwise and that ControlFALSE = 1 when treated and 0 when control.

The odds of preffering A when a person did not do so previously and did not receive a treatment (ControlFALSE=0 and PreferA=0) is $\exp(3.135)= 23$, i.e. there are 23 such persons who prefer A for every such person that prefers B. So A is very popular.

The effect of treatmeant refers to a person did not prefer A previously (PreferA=0). In that case the baseline odds decreases by a factor $\exp(-2.309) = .099$ or $(1-.099) \times 100\%=-90.1\%$ when she or he is subjected to the treatment. So the odds of choosing A for those who were treated and did not prefer A previously is $.099*23=2.3$, so there 2.3 such person who prefer A for every such person who prefers B. So among this group A is still more popular than B, but less so than in the untreated/baseline group.

The effect of prefering A previously refers to a person who is a control (ControlFALSE = 0). In that case the baseline odds decreases by a factor $.006$ or $-99.4\%$ when someone prefered A previously. (So those that pefered A previously are a lot less likely to do so now. Does that make sense?)

The interaction effect compares the effect of treatment for those persons that prefered A previously and those that did not. If a person prefered A previously (PreferA =1) then the odds ratio of treatment increases by a factor $\exp(2.850) = 17.3$. So the odds ratio of treatment for those that prefered A previously is $17.3 \times .099 = 1.71$. Alternatively, this odds ratio of treatment for those that prefered A previously could be computed as $\exp(2.850 - 2.309)$.

So the exponentiated constant gives you the baseline odds, the exponentiated coefficients of the main effects give you the odds ratios when the other variable equals 0, and the exponentiated coefficient of the interaction terms tells you the ratio by wich the odds ratio changes.

$\endgroup$
  • $\begingroup$ Thank you Maarten, this is very helpful as is your answer to my other, related question. I would just like a bit of clarification on one point, though. As I alluded in my other question, I am concerned about the statistical validity of what I have done here because of the fact that ControlFALSE has a high p-value in the first model and then a fairly low one in the second model. Applying your answer to my other question to this specific case, you said that this could happen if Control had a negative effect on one group of Prefer and a positive effect on the other. $\endgroup$ – Pygmalion Apr 24 '13 at 20:49
  • $\begingroup$ (ran out of space) Does that interpretation make sense here? I'm not exactly sure how to apply it directly. $\endgroup$ – Pygmalion Apr 24 '13 at 20:50
  • $\begingroup$ The effect of ControlFALSE in the first model is the effect of treatment for both those the prefered A previously and those that did not, while the effect in the second model is only the effect of treatment for those who did not prefer A previously. Whether that is OK or not is not a statistical question, but whether or not that makes substantive sense. $\endgroup$ – Maarten Buis Apr 25 '13 at 7:33
  • $\begingroup$ @MaartenBuis Great explanation. How would you do the equivent calculations for confidence intervals of the estimates? For ease of interpretation, I've generally stratified the logistic models (eg by prior preference in this example) and use the interaction term as a "statistical test for significant difference in OR. Is this acceptable? $\endgroup$ – bobmcpop Mar 2 '18 at 13:23
2
$\begingroup$

I also found this paper to be helpful in interpreting interaction in logistic regression:

Chen, J. J. (2003). Communicating complex information: the interpretation of statistical interaction in multiple logistic regression analysis. American journal of public health, 93(9), 1376-1377.

$\endgroup$
  • 4
    $\begingroup$ I have provided a full reference (title, author, date, journal etc) which means that contribution will still be useful if the link address changes. But could you expand on it to summarize the contents? Otherwise this is really more of a comment than an answer - we prefer our answers to be self-contained, so they are resistant to "link-rot". Alternatively we can convert this into a comment for you. $\endgroup$ – Silverfish Jul 25 '16 at 21:13
  • $\begingroup$ Thanks. I was linking NCBI so I thought it would be fine. I agree with the changes. Thanks! $\endgroup$ – deepseas Jul 25 '16 at 23:51
0
$\begingroup$

My own preference, when trying to interpret interactions in logistic regression, is to look at the predicted probabilities for each combination of categorical variables. In your case, this would be just 4 probabilities:

  1. Prefer A, control true
  2. Prefer A, control false
  3. Prefer B, control true
  4. Prefer B, control false

When I have continuous variables, I usually look at the predicted value at the median, 1st and 3rd quartiles.

Although this doesn't directly get at the interpretation of each coefficient, I find that it often lets me (and my clients) see what is going on in a clear way.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.