Appropriate test for proportion data

Question

I have two independent groups A and B. Each participant is given 5 questions to solve and his/her answer is evaluated as correct or incorrect.

However, participants can skip questions, so the number of attempted questions for each participant varies.

So, I end up with the data below:

     Group   # of correct answers  # of attempted answers   Correct Ratio
p1    A            3                       5                    0.6
p2    A            2                       4                    0.5
p3    B            1                       1                    1.0
p4    B            1                       3                    0.33
......

I want to test the whether the answers' correctness is different between group A and B. Which statistical test should I use?

Currently I simply count the total # of attempted answers and # of correct answers for all people from group A and B, say 90/100 vs 75/90, and use proportion test (in R as prop.test). But I don't know whether it is appropriate?

Answer 1

It depends on what you want to do with skipped answers. You can a) Assume they are questions the participant didn't know, and count them as errors (or, if the test is timed, this could be a penalty for slow work). b) Assume they would have been guessed at randomly (if the test is multiple choice). c) Assume nothing about them and ignore them. In a) the total number correct would be the score; in b) The total number correct + 1/p*m (where p is the number of choices and m is number of skipped answers) would be the score. In c) the score could be the proportion, as you have measured it.

In any of these, if you have only two groups and no other variables to add, you could do a t-test between groups, if the number of questions is pretty high. Technically, a) and b) would be counts, so t-test is not exactly appropriate, but, if there were enough questions, it would be OK.

Another alternative is regression. For a) and b) you could use a count model (Poisson or negative binomial regression). In c) you could use logistic, on the proportion of correct answers.