Stability of results in non-inferiority test in small sample size with rare events I am working with a dataset including 65 participants, 35 in the control group and 30 in the experimental group. The objective is to assess wether the experimental group is non-inferior to the control group.
There are 2 events in the control group and 1 in the experimental group. Independent sample test of proportions shows that the experimental group is non-inferior to the control group.
However, because of the rare event rate in combination with the small sample size, I want to measure the stability of our results.
Does anybody know how I can measure the stability of the results of the non-inferiority test in this dataset.
Thank you in advance,
Listeb
 A: First, I ask you to think about the logic behind the notion that your data shows non-inferiority even though, if just one more (adverse) event had occurred among the experimental group, that finding would have been reversed.  The difference between groups is almost the smallest possible difference.  To say that it supports any strong conclusion about a substantial difference in the population is counter-intuitive, right?  In one of your comments, if not in your original question, you seem to recognize this.  And when I run a one-tailed test of proportions, I obtain p = .32.
Second, I encourage you to answer @Henry’s questions.
Third, you can use a test of proportions to create a confidence interval (CI) for the difference you obtained between group percentages.  The CI’s soundness would depend on your method of sampling and on your choice of population to which you aim to generalize.  Right now I’m seeing that the difference is -0.024 (-2.4 percentage points) which under the right assumptions would yield a 95% CI of [-0.126, 0.079].  Such a CI would say something about the range of expected results on resampling, which I think translates to “stability.”
Fourth, from another comment I understand that you want to know what-size difference, with what sample sizes, would produce findings with a given p-value, or with a narrow CI.  You could learn this through trial and error with some software such as Excel; with a number of calculator websites; or working with a statistical power program such as the open-source G-Power.
