0
$\begingroup$

We ran a A/B test for all the customers and there are two sub groups within each group. How do I determine statistically conversion lift in one group is greater than conversion lift in other group.

For example. Control group (Higher income group, lower income group): out of 100 customers in each group, 20 and 50 purchased the product respectively. Conversion rate(Higher income - control) = 20%, Conversion rate(Lower income - Control) = 50%

Test group (Higher income group, lower income group): out of 100 customers in each group, 40 and 70 purchased the product respectively. Conversion rate(Higher income - Test) = 40%, Conversion rate(Lower income - Test) = 70% Conversion lift in Higher income group = (40 - 20)/20 = 100% Conversion lift in lower income group = (70 - 50)/50 = 40%

How to determine if the lift in higher income group is statistically significant?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

There are plenty of possibilities. E.g. one could probably calculate a logistic regression model with variables "high income group" (yes / no = low), "test group" (yes / no = control) and interaction "high income group" * "test group".

I would then test, if the interaction term is significant. If significant, one could conclude, that the effect of your intervention is different between groups.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks...I will build logistic regression model and check for significance of interaction effect. $\endgroup$
    – Saleem
    Commented Feb 28, 2020 at 17:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.