I'm conducting a study with the following ("hierarchical") design: There are two measures for each participant (2 time points, pre and post the study's intervention). There are control and test participants, for one of currently two different groups, but there will be more in the future.

So that's a 2-by-2-by-2 design:

  • 2 time points (pre-post)
  • 2 conditions (control-test)
  • 2 groups (group A-group B). There will be more groups in the future.

Additionally, I would like to control for some measured variable, called pred.

I have 3 questions:

  1. What is the "correct" statistical design to use? I'm struggling to choose between a linear mixed-effects model and using the differences in a multiple linear regression model.
  2. If LMM is the correct choice - should the "confounding" variable be considered a fixed effect?
  3. Given the choice in (1), how can I perform a power analysis to detect the required sample size? Given pilot data (for which I can fit the chosen model), how can I calculate the effect size and corresponding sample size? The required power is 0.8, and the alpha is 0.05.

Attaching a python code example to make it more clear, but answers are welcome to be written in R if it's more comfortable.

Linear Mixed Model

import pandas as pd
from statsmodels.formula.api import mixedlm

pilot_features = pd.read_csv("features.csv")
pilot_target = pd.read_csv("target.csv")

data = pilot_features.copy()
data["target"] = pilot_target["target"]

model = mixedlm("target ~ C(session, Treatment('pre')) * C(condition) * C(group) + pred",groups = data["participant"]).fit()

Multiple Regression

import pandas as pd
from statsmodels.formula.api import ols

pilot_delta_features = pd.read_csv("delta_features.csv")
pilot_delta_target = pd.read_csv("delta_target.csv")

data = pilot_diff_features.copy()
data["delta_target"] = pilot_diff_target["delta_target"]

model = ols("delta_target ~ C(condition) + C(condition):C(group) + delta_pred",data=data).fit()

There are several effects of interest:

  • Main effect of the condition.
  • Main effect of the session, only for test participants.
  • Interaction between condition and session
  • interaction between condition, session, and group.

1 Answer 1


Some points:

  • The model formulation will depend on the specific questions of interest (i.e., which main effects and interactions to include).
  • If you have missing data in your outcome variable target it is preferable to use a linear mixed model.
  • Working with differences (i.e., target at post minus the target at pre) is, generally, considered suboptimal. It is better to use an ANCOVA model (i.e., linear regression), in which you put as the outcome the target at post, and you include as a covariate the target at pre.
  • $\begingroup$ Thank you Dimitris. * Regarding the questions of interest, I added a list of interesting contrast at the bottom of the post - do these contrasts emerge from the formula I suggested at the top? * I don't have missing data in my outcome variable. * Thank you for your suggestion - so, between linear mixed models and ANCOVA - which would you suggest I use for my analysis? Thank you! $\endgroup$
    – Gal Kepler
    Mar 26, 2023 at 7:22
  • $\begingroup$ If you do not have missing data, the ANCOVA approach could be used here. $\endgroup$ Mar 26, 2023 at 19:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.