I am running a test across two groups over a period of time and want to understand if the Change in proportions for my test group is significantly different to the change in proportions in the control group.

Most of what I have found compares one proportion to another either over group or over time but not both.

If I have: my Test group with proportions P1 and P2 with sample N1 and N2 (in periods T1 and T2) my control group with proportions Q1 and Q2 with sample M1 and N2 (in periods T1 and T2)

I want to know if P2-P1 is statistically significantly different to Q2-Q1

P1,P2,Q1, Q2 are all percentages.

Any help would be most appreciated!


1 Answer 1


So assuming N1P1 is sufficiently large (and similarly for the other pairs, where I assume the proportions are for the rarer event...eg making vs not making a sale), then you can approximate the distribution of proportions by a normal.and use the fact that a linear combination of normals is also normal.

So then you calculate the variance of z=(P2-P1) - (Q2-Q1). Assuming the proportions are independent, the variances add up, ie $var(z) =var(P2) +var(P1) +var(Q2) +var(Q1) $ So then you perform a z test on the variable z, (with assumed mean under the null hypothesis of zero).

See the normal approximation section of https://en.m.wikipedia.org/wiki/Binomial_proportion_confidence_interval which discusses the limits of the approximation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.