This question seems so basic I am almost embarrassed to ask it, but my need for clarity has finally exceeded my need to pretend like I know everything.

I am testing the effect of prior training on the ability of two groups to learn from an intervention. Learning is measured by correct response to a task, before and after an intervention (factor time: baseline = 0, followup = 1). Prior training is a two-level factor: no trained = 0 vs trained = 1.

I want to know how to obtain the estimated difference in log-odds between baseline and follow-up for the training group. Here is the output for the repeated-measures logistic regression.

                             Estimate Std.Err z-value    p-value
(Intercept)                    0.4726  0.2701  1.7495 0.08020486
trainingtrained                1.6864  0.6563  2.5694 0.01018619
timefollowup                   2.5595  0.7607  3.3648 0.00076599
trainingtrained:timefollowup  -1.3892  1.2549 -1.1070 0.26829049

The simple effects of time, i.e. the difference in log-odds between baseline and followup in the no training group are easy enough, it is just the coefficient for timefollowup = 2.5595.

What I am unsure about is: how to calculate the simple effects of time in the training group only?

My guess is that you add the trainingtrained and trainingtrained:timefollowup coefficients together, i.e. 1.6864 + -1.3892 = 0.2972, but I just wanted to check. A little voice is telling me that the intercept needs to be involved somehow but I don't know whether that voice is right or not.


You can just change what the reference level is for the training variable by using relevel() and refit the model. When you do so, the coefficient on timefollowup will be the simple effect of time at the no trained level of training.

If for some reason you can't refit the model (e.g., because the dataset is huge), you can use contrasts to compute the simple effect and its standard error. The simple effect at trained is the sum of the coefficients on timefollowup and trainingtrained:timefollowup. This is because the coefficient on trainingtrained:timefollowup represents the difference between the simple effect of time at training = trained and the simple effect of time at training = no trained (the latter of which is the coefficient on timefollowup). The intercept doesn't need to be involved because you're not interested in the log odds at any level of the variables; you're interested in the change in log odds, which the intercept is not involved in (i.e., if you computed the model-implied log odds at each level of the covariates and took the differences among them, the intercept would always cancel out).

To use contrasts, create a matrix [0 0 1 1]. Then multiply this matrix by the vector of coefficients to get the effect of interest. To get its standard error, you need to pre- and post-multiply the covariance matrix of the coefficients (i.e., extracted using vcov(fit)) and take the square root of the computed value.

C <- matrix(c(0, 0, 1, 1), nrow = 1)
simple_effect <- C %*% coef(fit)
simple_effect_se <- sqrt(C %*% vcov(fit) %*% t(C))

You can also use glht() in the multcomp package which performs these actions for you.

summary(glht(fit, "timefollowup + trainingtrained:timefollowup = 0"))

(You can also replace the part in quotes with the C matrix.)

| cite | improve this answer | |
  • $\begingroup$ Thank you @Noah. So I guessed incorrectly about the effects of time in the trained group? I guessed it was obtained by adding trainingtrained and trainingtrained:timefollowup coefficients together, but you said in your answer that it is obtained by adding timefollowup and trainingtrained:timefollowup together. So does that mean that the sum of the adding trainingtrained and trainingtrained:timefollowup coefficients represents the simple effect of training at followup? $\endgroup$ – llewmills Mar 30 at 9:24
  • $\begingroup$ Yes, that is correct. $\endgroup$ – Noah Mar 30 at 17:05

The contrast matrix approach works very well. But if you are struggling with the concept, one more basic approach is to use a plug-in technique. We can actually write out the model fit to your data as:

$$ log[Odds(learning)] = 0.47 + 1.68\times trained + 2.56\times time - 1.39\times trained \times time $$

You asked about the estimated difference in log-odds between baseline and follow-up for the training group. In modeling terms, this is equivalent to setting all $time $ values to 1 and all values of $trained $ to 1, and then setting all $time$ values to 0 and all values of $trained $ to 1, and taking the difference of log-odds that results from these two equations:

\begin{split} \left ( 0.47 + 1.68\times \color{red}{1} + 2.56\times \color{red}{1} - 1.39\times \color{red}{1} \times \color{red}{1} \right ) \\ - \left ( 0.47 + 1.68\times \color{red}{1} + 2.56\times \color{red}{0} - 1.39\times \color{red}{1} \times \color{red}{0} \right ) \end{split}

Doing this basic math gives us a difference in the log-odds of learning for those who were trained of:

$$ 2.56\times \color{red}{1} - 1.39\times \color{red}{1} \times \color{red}{1} = 1.17, $$

which can be translated to an odds ratio, if you wish, by exponentiating. This is actually what is happening with the contrast matrix multiplication approach described, it's just easier to understand if you aren't too familiar with matrix multiplication. The drawback of this basic approach is that calculating standard errors is a bit more cumbersome, whereas it can be easily done with contrast matrices.

| cite | improve this answer | |
  • $\begingroup$ Thanks you @user1849779 yes it was the standard error and p-values I was struggling with. Appreciate the suggestions. $\endgroup$ – llewmills Apr 3 at 19:58

It sounds weird since adding the coefficients of trainingtrained and trainingtrained:timefollowup implies that you consider also the interactive effect of trainingtrained and timefollowup on the response when both of them have the value of 1. However, it means that you are investigating not only the effect of training group, but also a part of the effect due to these two predictors.

If you want simple effect of time in the training group only, you should just take the value $1.6864$ into account.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.