0
$\begingroup$

I have data from an intervention study on math learning. Participants were assigned to four treatment conditions in a 2*2 between-participants design. Participants were tested before the intervention (pretest), during the intervention (practice), and twice after the intervention (immediate and delayed posttest).

Edit: The raw data are accuracy (0 or 1) for each trial for each test point for each participant, like this:

ID        factor_1  factor_2  section   trial     accuracy
0001      cond_A    cond_X    pretest   1         0
0001      cond_A    cond_X    pretest   2         1
...       ...       ...       ...       ...       ...
0001      cond_A    cond_X    practice  1         1
0001      cond_A    cond_X    practice  2         1
...       ...       ...       ...       ...       ...
0001      cond_A    cond_X    imm_post  1         1
0001      cond_A    cond_X    imm_post  2         1
...       ...       ...       ...       ...       ...
0001      cond_A    cond_X    del_post  1         1
0001      cond_A    cond_X    del_post  2         1
...       ...       ...       ...       ...       ...
0002      cond_A    cond_Y    ...       ...       ...
...       ...       ...       ...       ...       ...
0003      cond_B    cond_X    ...       ...       ...

The data used in my current analyses are accuracy (proportion correct) collapsed across trials for each test point for each participant, with condition factors centered, like this:

ID        factor_1  factor_2  pretest   practice  imm_post  del_post
0001      1         1         .70       .90       .85       .80
0002      1        -1         .71       .91       .86       .81
0003      -1        1         .69       .89       .84       .79
...       ...       ...       ...       ...       ...       ...

The ranges of accuracies in the four conditions for the four test points were as follows.

  • Pretest: [65%, 71%]
  • Practice: [94%, 97%]
  • Immediate posttest: [89%, 91%]
  • Delayed posttest: [79%, 84%]

I submitted accuracies on practice, immediate posttest, and delayed posttest to Bayesian mixed linear regression (using brm from the brms package) with pretest accuracy, the two condition factors, and the interaction of the two condition factors as fixed effects and participant as a random effect. I found no effects involving the condition factors in any of these analyses. Because practice and immediate posttest accuracies were high in all conditions, the absence of condition effects for these test points might reflect ceiling effects.

How can I assess whether the null effects of condition reflected ceiling effects? Ideally, I would like a statistical test of some sort.

$\endgroup$
3
  • 1
    $\begingroup$ Can you share a sample of the data, even if it is fabricated? I want to be sure we're on the same page when it comes to how the outcome is being represented. $\endgroup$ Commented Sep 20, 2023 at 14:05
  • $\begingroup$ @DemetriPananos, done, thank you! $\endgroup$
    – baixiwei
    Commented Sep 20, 2023 at 15:04
  • 1
    $\begingroup$ Looks like the outcome is binomial. Why not use a binomial family in brms? That way, there are no ceiling effects (on the log odds scale) $\endgroup$ Commented Sep 20, 2023 at 15:14

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.