2
$\begingroup$

I ran a study with two IVs, a between-subjects manipulation Hands vs NoHands, and a within-subjects manipulation High and Low. The outcome of what participants did was recorded and then show to one Rater. This resulted in a binary outcome of 1=correct, 2=incorrect for each statement made by the participants.

But I have two issues with this: 1. each participant has 4 responses (2 High, 2 Low), so there is dependency in the responses, which I don't know how to address. 2. Since it is a mixed design, neither Chi-square or McNemar work.

What is the correct analysis I can use on this data? Please help

Improve this question
$\endgroup$
3
  • $\begingroup$ Can you clarify your research question and what you believe will happen? The study design is not the only consideration in choosing an analysis approach. $\endgroup$
    – Hotaka
    May 21, 2015 at 16:46
  • $\begingroup$ Thank you all for the useful information. Summing up the responses per subject would account for the dependency issue, but would change the information it provides, as I care more about the overall accuracy of the rater for each response. $\endgroup$
    – Victor555
    May 27, 2015 at 14:17
  • $\begingroup$ Just a small clarification, the study data in this case was an N-of-1 design, as there is a single rater making all the accuracy decisions. I just need a way to account for the dependency of the observations. otherwise I just run a chi-square and im done, but it violates the assumption of independence. So I need something like a covariate but for a chi-square (or in my case a Fisher's exact test). $\endgroup$
    – Victor555
    May 27, 2015 at 14:19

1 Answer 1

Reset to default
0
$\begingroup$

In waiting for a better answer, I would advocate for a sum the outcome over the IV you don't want to analysis in order to have only one measure for each subject. You can then use the Wilcoxon paired test.

Let's say you want to analyse the effect of Hands vs NoHands over the outcome. I would sum the outcome of High and Low for each subject. Then, for each subject and each condition you have a value going from 0 to 2. Wilcoxon paired test should work there, but I have to mention that I do not know how powerful this test is when there are only 3 levels. One alternative might be to gather 2 levels (like 0 and 1) and use the MacNemar test.

As you want to test both IVs, you might want to use a correction for multiple testing like the Holm–Bonferroni_method

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ I will need more information from the OP, but I don't think this is a good idea because you are losing information by summing. If you have a 1 out of 2, you do not know which of the two responses the subject got correct. $\endgroup$
    – Hotaka
    May 21, 2015 at 16:49
  • $\begingroup$ Indeed it completely depends on the goal of his study. Depending on his aim, losing this information might not a big deal though. $\endgroup$
    – brumar
    May 21, 2015 at 17:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.