# How to test for differences in binary score data?

I have data from a human subject study that was setup as a 3x2 (system by task type) experiment, with 24 participants. Each participant completed 6 tasks in all, one using each of the 3 systems, and 2 task types. I have two performance measures: the first is completion time, and the second is a human assigned completion score. The score is either 1 if the task was completed correctly or 0 if the task was completed incorrectly

I am interested in seeing in there are differences between systems overall, and within each of the task types. So, for the time data I have used a basic RM-ANOVA. However, I am puzzled about how I should treat the score data. On the one hand it is binary score, and because there is only one trial in each system by task type combination the distribution can't be normal. On the other hand, it is score data, such as the results from a true and false test, and so why should it be treated differently than if score had 100 questions (in which case I could run and RM-ANOVA, right?).

I think I missing a basic assumption here.

-
I think the basic assumption is that your response variable isn't continuous. I don't feel like this warrants a full answer, but there are a couple things you could try. If want to look at was the interaction between system type and task, you can do a goodness-of-fit test (chi-squared or fisher's exact) on the count data. This works well in a 2x2 design but it might be harder to draw conclusions on a 3x2 design. The other thing is you could try is fitting a logistic regression model, which effectively transforms your binary response variable into a continuous one (a probability of success). –  Oliver Jul 13 '12 at 17:45
I should note that I meant my comment doesn't warrant being written as an answer, not that the question doesn't deserve an answer* –  Oliver Jul 13 '12 at 19:02
Thanks for your thoughts @Oliver! I will look into those options. –  batmanfu Jul 13 '12 at 19:20
Based on @Oliver 's comments, I have looked into chi-square (but it has an assumption of independence) and fisher's exact (2x2 contingency table). Logistic regression seems like it might be a possibility. I also came across tau-c, which allows testing of ordinal data for rectangular contingency tables (like I would have), I wonder if it might be appropriate? –  batmanfu Jul 13 '12 at 21:37
add comment

## migrated from math.stackexchange.comJul 13 '12 at 5:12

This question came from our site for people studying math at any level and professionals in related fields.