5
$\begingroup$

I'm looking for a method to determine the likelihood that two percentages from independent groups are EQUAL. In this case I'm looking at the likelihood of symptom improvement from patients gathered through two different methods (one traditional double-blind study and one from an online enrollment).

So if in the traditional method 60% of 169 patients reported improvement and in the online study 63% of 110 patients reported improvement. Is there a way to tell whether these two percentages are the same? I know I can use a chi-squared test to determine the likelihood that they are different but can I use 1 minus chi-squared as the likelihood that they are the same?

Is there a more appropriate test I can do?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ The p-value from the chi-square test does not tell you the likelihood that they are different. The p-value tells you what the probability is of the observed difference in the percentages or an even more extreme difference under the assumption that there really is no difference in the true percentages. That is something very different than the probability that the true percentages are different. $\endgroup$
    – Wolfgang
    Commented Sep 7, 2011 at 22:59
  • $\begingroup$ In science you can't prove anything. $\endgroup$
    – suncoolsu
    Commented Sep 8, 2011 at 0:19
  • $\begingroup$ @Wolfgang ... I understand that the chi-squared tells me the likelihood of finding values being this different given the assumption that they from the same distribution. I was unclear (and leaning towards no) as to whether a 1-p could be used to say that they truly were from the same distribution. $\endgroup$
    – JudoWill
    Commented Sep 8, 2011 at 5:12
  • $\begingroup$ Your "leaning towards no" is correct. 1-p will be the probability of the observed data or less extreme data under the assumption of the null hypothesis. Again, that is different than the probability that the two percentages are the same. $\endgroup$
    – Wolfgang
    Commented Sep 8, 2011 at 11:24

1 Answer 1

Reset to default
4
$\begingroup$

What you are wanting to do is called an equivalence test. They are not very common, and are even easier to misunderstand than a conventional hypothesis test because the roles of false positive and false negative errors are switched around.

Harvey Motulsky's website has a good basic description: http://www.graphpad.com/library/biostatsspecial/article_182.htm

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ perfect, just what I was google-ing for but couldn't find :) $\endgroup$
    – JudoWill
    Commented Sep 8, 2011 at 5:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.