0
$\begingroup$

I want to run a A/B test where option B is more profitable than option A. Option A has a response rate of 70%. Hence i want to achieve a response rate of 70% for both B as well. I want no statistical difference between the two response rates. Lets say i ran N cases through both A and B and received 70% response rate in both cases. I want to conclude that there is no statistically significant difference between the two proportions.

Edit:-Any difference less than 50 bps(0.5%) would be considered immaterial and i want to draw the conclusion at 95% confidence and 80% power

What should be the minimum N and how would i determine it?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ That's not a conclusion you can make. All you can say from such data is that you cannot detect a difference of some specified size. Thus, to ask a question like this, you need to stipulate (a) how small a difference you consider to be immaterial and (b) the confidence with which you want to draw your conclusion. $\endgroup$
    – whuber ♦
    Commented Dec 6, 2017 at 18:27
  • 1
    $\begingroup$ Any difference less than 50 bps would be considered immaterial and i want to draw the conclusion at 95% confidence and 80% power. What should be the minimum sample size now? $\endgroup$
    – IamNotLegend
    Commented Dec 6, 2017 at 21:01
  • $\begingroup$ A similar question: stats.stackexchange.com/questions/235750/… $\endgroup$
    – kjetil b halvorsen ♦
    Commented Dec 6, 2017 at 21:26
  • $\begingroup$ It doesn't answer my question. $\endgroup$
    – IamNotLegend
    Commented Dec 6, 2017 at 21:31
  • $\begingroup$ Agreed--it answers a different question. Your improved formulation is excellent (although it would be good to spell out what "bps" is--not everyone will interpret it as basis points or even know what a basis point is). Please incorporate that information in your question (not everyone consults the comments). $\endgroup$
    – whuber ♦
    Commented Dec 6, 2017 at 22:17

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.