Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 7 down vote favorite
1

I'm relatively new to statistics, and understand that my question may be completely misworded. I am testing my own algorithm versus another. While the outputs are not identical, I want to show that the differences are "statistically insignificant." How can I quantify this, to make my point?

share|improve this question
That'll depend on what kinds of data you're dealing with, and how big your samples are. Can you edit your post to include a more detailed answer? – naught101 May 22 '12 at 2:32
The equivalence test is powwered to reject nonequivalence. It forces you to pick the sample size large enough to have power. Without reversing the null and alternative hypothesis in small samples you would have very little change of rejecting the null hypothesis of no difference. But not rejecting is not the same as accepting because of the lack of power, That is why Blackwelder makes non-equavalence the null hypothesis and showing equivalence requires rejecting the null. – Michael Chernick May 22 '12 at 3:23
Note that the null hypothesis is that the difference in means is greater that a specified delta (the window of equivalence). – Michael Chernick May 22 '12 at 3:23

1 Answer

up vote 11 down vote accepted

If you are comparing two groups and want to show no significant difference, this is called equivalence testing. It essentially reverses the null and alternative hypotheses. The idea is to define an interval of insignificance called the window of equivalence. This is used a lot when trying to show that a generic drug is a suitable replacement for a marketed drug. A good source to read about this is William Blackwelder's paper titled “Proving the null hypothesis” in clinical trials.

share|improve this answer
Thank you Michael. I guess I need to do some more work before I can declare the data "significant" or "insignificant." I will start looking into this, and let you guys know if I have any follow-up questions, which I'm sure I will. – Adam_G May 22 '12 at 11:40
3  
Just one last comment. The one-sided version of equivalence is called noninferiority. Often drug manufacturers can bring a so-called "me too" drug to market that does not even have to be as good in efficacy as the marketed competitor as long as it is not much worse (defined by a delta that is a window for noninferiority. This can be done because the product offers some other advantage to the consumer, like fewer doses per day or as in the case of an anticoagulant not needing to have INR monitored. – Michael Chernick May 22 '12 at 13:28

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.