I have a data set that is a group of participants who accept or reject each of two devices, and I'd like to test if the two devices are accepted at different rates. Summary table like so

Overall Acceptance
        Accept  Reject
X       124     20
Y       111     33

What's the best statistical test to determine if the difference between X's acceptance rate and Y's acceptance rate is significant? I'm unused to binary data so out of my depth here.

  • $\begingroup$ You are looking for logistic regression stats.stackexchange.com/questions/tagged/logistic+regression $\endgroup$ – B Williams May 28 '13 at 16:29
  • $\begingroup$ I get the impression from some of your responses under my answer that you're in communication with a third party, perhaps a supervisor or some such, relaying answers and responses to them. Is this the case? $\endgroup$ – Glen_b -Reinstate Monica Jun 5 '13 at 4:58
  • $\begingroup$ Not quite--I'm working with a market research team who occasionally request or suggest things, but as analysis isn't their primary expertise, they generally leave me with partial info that needs filling-in or clarification. $\endgroup$ – Krysta Jun 5 '13 at 16:58

There are several options.

(i) You could do a two-sample test of binomial proportions / two sample proportions test.

With your sample size, the normal approximation should be okay, though - you don't necessarily have to worry about the binomial part.

(ii) You could do a chi-square test of independence (which also tests equality of proportion); this is basically equivalent to the first option if your test is two-tailed, or similarly, you could do a $G^2$ test.

(iii) You might do a Fisher test, I guess.

(You could do something more complicated like a logistic regression but I don't see the need here.)

Depending somewhat on your area, the 2x2 chi-square test is probably the most likely to be familiar to other people looking at it. If you want a one tailed test, the two sample proportions test is the way to go.

  • $\begingroup$ My problem with doing a 2x2 chi-squared is that it's likely to tell me that the accept vs reject numbers are different; that's fine, but what I really want to know is whether the X vs Y accept numbers are different. $\endgroup$ – Krysta May 29 '13 at 13:21
  • $\begingroup$ You are mistaken. The 2x2 chi-square actually conditions on the margins and compares the proportions (that is, one of two things it doesn't do it test the accept vs reject margin); there are four different proportions comparisons that all correspond to exactly the same chi-square value, including the one you want. If this is not clear to you then I urge you to do it explicitly as a two sample proportions test and present it that way (and for your own understanding, then do it as a chi-square and see you get the same p-value as long as you treat other considerations the same). $\endgroup$ – Glen_b -Reinstate Monica May 29 '13 at 21:44
  • $\begingroup$ I didn't articulate my point well; the problem with a chi-squared test is that the results don't differentiate between Accept vs Reject being different, and X vs Y being different. Telling me that overall a combination of the two are different doesn't answer the question, so I need a different test. $\endgroup$ – Krysta Jun 3 '13 at 15:55
  • $\begingroup$ You are confused. Would you agree that a straight difference in the Accept proportions for X and Y measures the difference you want to test? $\endgroup$ – Glen_b -Reinstate Monica Jun 3 '13 at 16:05
  • $\begingroup$ To test that, it's important to standardize (divide by its standard deviation). You'd agree that such scaling doesn't alter which cases are the most extreme (the ones you want to reject)? If you're doing a two tailed test, the extremes correspond to the Accept proportion for X being much larger than for Y, and the other way around. Everything okay so far? $\endgroup$ – Glen_b -Reinstate Monica Jun 3 '13 at 16:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.