Let's say I ran an experiment where 100 people performed 4 identical tasks (Task 1-4) on two different software packages (Package A vs. Package B) such that I have 8 trials-worth of data for each participant:
I have two measures: objective success (i.e., Did they complete the task successfully, 1=yes, 0=no) and subjective success (i.e., Did they think they completed the task successfully, 1=yes, 0=no).
I believe these data can be summarized in a contingency table showing something like the count of:
- Hits: Obj. success = 1 & Subj. success = 1
- Misses: Obj. success = 1 & Subj. success = 0
- False Positives: Obj. success = 0 & Subj. success = 1
- Correct Rejections: Obj. success = 0 & Subj. success = 0
for each Software Package used at each level of Task.
What I'd ultimately like to understand is whether there are significant differences in the rate of misses and false Positives between packages A and B on each task. I'm not sure whether such interactions can be tested using a chi-square or logistic regression, or perhaps something else, and I had hoped some of the good people on this forum might be able to enlighten me.