3
$\begingroup$

I want to run a mixed-effect regression model on a few data points. I have 24 participants and 4 trials per participant. I want to include two fixed effects and their interaction in the model, as well as random intercepts of participants and trials. Will the sample size be enough?

One fixed effect is condition (experimental/control), and the other one is task version (A/B). Half of the participants saw four trials - i.e., experimental A trial (x2) and experimental B trial (x2) - and the other half saw control A trial (x2) and control B trial (x2). Can I perform a mixed-effect logistic regression? Or should I choose another method?

Thank you very much!!!

Improve this question
$\endgroup$
2
  • $\begingroup$ You can also consider changing the experimental design so that every participants performs one trial under all four combinations (A0, A1, B0, B1 where 0 and 1 indicate control and experimental respectively). This design will give smaller std. error for the condition effect (control vs experimental) with the same number of participants. $\endgroup$
    – dipetkov
    Commented May 10 at 10:54
  • $\begingroup$ Unfortunately, I can't do that, but it's a smart trick to remember for the future! $\endgroup$
    – chiaras15
    Commented May 10 at 16:33

2 Answers 2

Reset to default
5
$\begingroup$

Look at this for a discussion of the minimum number of subjects in a binary logistic regression analysis. It argues that the bare minimum number is $n=96$ due to the low-information nature of binary response variables. You are pushing the envelope quite a lot. With your sample size you need to have a high-resolution continuous response variable with high test-retest reliability.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Hi, thank you! Very helpful resource :) So you don't think the number of trials matters as well? I will definitely have to change the method if I rely only on the number of participants $\endgroup$
    – chiaras15
    Commented May 9 at 18:57
  • $\begingroup$ The effective sample size is what matters. For your situation the effective $n$ is probably around 36-48. It depends on within-subject correlation. $\endgroup$
    – Frank Harrell
    Commented May 9 at 22:56
4
$\begingroup$

I agree that your sample is way too small for mixed effects logistic regression, let alone a regular logistic regression. But you can use a much simpler model.

You can run a linear regression of the outcome on your two fixed effects and a third fixed effect for trial (assuming there are only 4) and use a cluster-robust standard error to adjust for individual observation. Even though your outcome is binary, the cluster-robust standard error adjusts the inferences to be approximately normal. Now your coefficients can be interpreted in terms of the probability of selecting the outcome rather than logs odds.

In R, you would run the analysis as so:

fit <- estimatr::lm_robust(outcome ~ condition * version + trial, data = data,
                           clusters = participant_id)

You can use marginaleffects::avg_predictions() to compute condition-specific predictions and their contrasts as a way to probe the model.

Note that including trial as a predictor is unnecessary; it can either increase or decrease your statistical power depending on how much variation it explains in the outcome. I would actually recommend excluding it with such a small sample.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.