# Non-inferiority margin and minimum detectable effect vs sample size

I am trying to understand why testing for non-inferiority requires pretty much the same sample size as testing for superiority (I assume the latter is the same as a one-sided test for a given MDE).

I was asked how big a sample we need to test that a certain change to our website's backend has no (negative) effect on visitor conversion. I said that it should be easier than testing for a lift in conversion. But that doesn't seem to be the case.

Running a one-sided test with 95% confidence and 90% power and assuming a 9% conversion rate and a 5% effect (0.45% lift) requires some 70k examples: http://powerandsamplesize.com/Calculators/Compare-2-Proportions/2-Sample-1-Sided

At the same time, running a non-inferiority test with a 0.45% margin requires 69k samples http://powerandsamplesize.com/Calculators/Compare-2-Proportions/2-Sample-Non-Inferiority-or-Superiority

Is that right or am I missing something?

• The table in the blog assumes that there is in truth an improvement of 5 percentage points and under that assumption non-inferiority with a margin of 2 percentage points is easier to show than superiorty. Very approximately non-inferiority is as easy to show as superiorty assuming that the true difference is 5+2=7 percentage points (and sample size is about proportional to effect size squared i.e. $5^2/7^2\approx 0.5$). Apr 2, 2018 at 5:21