I have data from an intervention study on math learning. Participants were assigned to four treatment conditions in a 2*2 between-participants design. Participants were tested before the intervention (pretest), during the intervention (practice), and twice after the intervention (immediate and delayed posttest).
Edit: The raw data are accuracy (0 or 1) for each trial for each test point for each participant, like this:
ID factor_1 factor_2 section trial accuracy
0001 cond_A cond_X pretest 1 0
0001 cond_A cond_X pretest 2 1
... ... ... ... ... ...
0001 cond_A cond_X practice 1 1
0001 cond_A cond_X practice 2 1
... ... ... ... ... ...
0001 cond_A cond_X imm_post 1 1
0001 cond_A cond_X imm_post 2 1
... ... ... ... ... ...
0001 cond_A cond_X del_post 1 1
0001 cond_A cond_X del_post 2 1
... ... ... ... ... ...
0002 cond_A cond_Y ... ... ...
... ... ... ... ... ...
0003 cond_B cond_X ... ... ...
The data used in my current analyses are accuracy (proportion correct) collapsed across trials for each test point for each participant, with condition factors centered, like this:
ID factor_1 factor_2 pretest practice imm_post del_post
0001 1 1 .70 .90 .85 .80
0002 1 -1 .71 .91 .86 .81
0003 -1 1 .69 .89 .84 .79
... ... ... ... ... ... ...
The ranges of accuracies in the four conditions for the four test points were as follows.
- Pretest: [65%, 71%]
- Practice: [94%, 97%]
- Immediate posttest: [89%, 91%]
- Delayed posttest: [79%, 84%]
I submitted accuracies on practice, immediate posttest, and delayed posttest to Bayesian mixed linear regression (using brm from the brms package) with pretest accuracy, the two condition factors, and the interaction of the two condition factors as fixed effects and participant as a random effect. I found no effects involving the condition factors in any of these analyses. Because practice and immediate posttest accuracies were high in all conditions, the absence of condition effects for these test points might reflect ceiling effects.
How can I assess whether the null effects of condition reflected ceiling effects? Ideally, I would like a statistical test of some sort.