What type of statistical test should be used to compare successes of two treatments?

Say I have two treatments, "hot" and "cold". I care about a certain outcome, say "success". I have data on the number of successes out of, say 100 trials in the "hot" treatment. Similarly, I have data on the number of successes out of 100 trials in the "cold" treatment.

What sort of statistical test should I use to determine whether there is any significant difference between my two treatments with respect to success rate?

• Was treatment assignment by randomization or up to the treating physician? Any other factors offer information that may determine outcome or assignment available? – Björn Jun 10 '17 at 16:52
• Treatment category was assigned at random. Does that clarify? – Atticus29 Jun 10 '17 at 16:56

This is a basic 2 by 2 frequency table. A chi-square test of independence would be appropriate. Also, you could use a risk ratio or odds ratio to describe the size of the effect and a construct confidence interval around whichever one you choose.

You have two treatments: drug A (hot) and drug B (cold). The Null- Hypothesis would be that cure rate (or proportions) of A equals to the cure rate of B.

Null hypothesis: $p_{A} = p_{B}$

Alternative hypothesis: $p_{A} != p_{B}$

You could use z-test for proportions: $z = \frac{p_{A} - p_{B}}{\sqrt{p(1-p) (\frac{1}{n_A} + \frac{1}{n_B}})}$

And then check the p values.

In python you could use proportions_ztes