Suppose I am testing two drugs, I and II. The response profile is either I > II, I < II or I = II. I was trying to figure out if I is in fact better than II.

One way I thought of was to throw out all the ties (I == II), and then assign a 1 where I > II and a 0 where I < II. Then this boils down to a binomial distribution (which I can approximate with the normal).

But I was wondering if throwing away the ties is really appropriate... If I don't throw away ties, I can't use the binomial distribution (so would I switch to some multinomial test)?

The other thing I can think of is some wilcoxon rank sum test or mann whitney u test with ties... but I was wondering if anyone else has an idea? The concern here is that there will be a LOT of ties...


  • $\begingroup$ Are the data really that coarse? Are there really only 3 states, or are there, for example, instances when I > II and then instances when I >> II? $\endgroup$ – A.M. Aug 16 '13 at 3:30
  • $\begingroup$ Hm, for now, let's assume it's really that coarse... $\endgroup$ – user1357015 Aug 16 '13 at 4:15

It's still binomial even after throwing out the ties, and you get a more powerful test that way. But for describing the size of the difference, you should keep the ties and present the three sample proportions.

  • $\begingroup$ It's easy to come up with proportions but I would need confidence intervals? How would I do that? Use the binomial for each proportion and then take a multiple comparison adjustment? $\endgroup$ – user1357015 Aug 16 '13 at 5:13
  • $\begingroup$ The CI for the conditional probability Pr(I > II | I <> II) is obtained the usual way, using only the untied data. The trinomial case is messier because it's a 2-dimensional region, not a simple 1-dimensional interval, and you need to know who you're talking to and how many technical corners you need to round to be understood (or, more importantly, to not be misunderstood). $\endgroup$ – Ray Koopman Aug 16 '13 at 7:03
  • $\begingroup$ So what you're saying is first do a test to see if I == II -- easy enough, a chi-square test. Now for Pr(I > II | I < II), your saying throw out all the ties and check. How are you certain that throwing away the ties is the right way to proceed... your n would go down which would effect the result. Sorry if I'm missing something obvious. $\endgroup$ – user1357015 Aug 16 '13 at 11:15
  • $\begingroup$ Your report might read something like this: "Responses to the drugs differed in x% of the cases (95% CI = (x0,x1)). When they differed, the response to I was better than the response to II in y% of the cases (95% CI = (y0,y1))." If A, B, and C are the counts for I > II, I < II, and I = II then x = 100(A+B)/(A+B+C) and y = 100A/(A+B), and the CIs follow accordingly. $\endgroup$ – Ray Koopman Aug 16 '13 at 16:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.