I would like to compare two error rates (proportion of incorrect answers in questions with dichotomous response-format) for two different items, and I am looking for some advice on how to do that.
Control Group (CG) N = 100
Experimental Group (EG) N = 120
Item A: was answered incorrectly by 20 out of 100 participants in the CG (= 20%) and by 50 out of 120 participants in the EG (= 42%). --> I can calculate a chi-squared test whether the proportion of incorrect answers is higher in the EG than in the CG.
Item B: was answered incorrectly by 33 out of 100 participants in the CG (= 33%) and by 80 out of 120 participants in the EG (= 67%). --> Again, a chi-squared test can be calculated.
I would like to calculate whether the increase in the error rate is more pronounced for Item A (diff 33% to 67% = +34%) than for Item B (diff 20% to 42% = +22%).
EG and CG were manipulated between subjects; all items were answered by all participants, i.e., manipulated within subjects.
My idea is that I have to take into account the differences in the baseline errors (CG). So I thought about taking the baseline errors as 100% and compare the increase in the EG for Item A (+250%) to the one in Item B (+242%).
I'd be very grateful for any comments on whether this procedure is correct/makes sense in principle, and, if so, how the two values may be compared. I'm also unsure whether I still have to take into account that the data are kind of dependent as both items were answered by all participants?