# Sample size required to determine which of a set of advertisements has the highest click through rate

I am a software designer by trade and I am working on a project for a client, and I would like to make sure that my analysis is statistically sound.

Consider the following: We have n advertisements (n < 10), and we simply want to know which ad performs the best. Our ad server will randomly serve one of these ads. Success is if the a user clicks on the ad -- our server keeps track of that.

Given: Confidence Interval: 95%

Question: What is the estimated sample size? (How many total ads must we serve), Why? (remember i am a dummy)

Thanks

• Could you clarify what you means by "margin of error 5%"? – onestop Mar 10 '11 at 20:12
• @onestop -- good clarification -- I removed it from the question. I just took that variable from the following sample size calculator: raosoft.com/samplesize.html But I don't think its relavant in this question. Thanks! – Jonathan Mar 10 '11 at 22:06
• For many tests, you can calculate a sample size such that the test achieves a certain power given an assumed (fixed) effect size. In other words, you have to specifiy these things first: 1) what test do you want to use? 2) what power do you want that test to have? conditional on 3) an effect size that you deem interesting. 1) is something people here can probably help you with. 2) might be related to the 95% you indicated. 3) however, is something you have to provide beforehand: how different do the probabilities have to be to be considered interestingly different? – caracal Mar 11 '11 at 11:16
• So if i have to give more parameters here you go: 1. test to use -- no idea -- do you have suggestions? 2. power: even after looking at the wikipedia definition - I do not know how to intelligently answer that. 3. effect size: Lets say 10% better – Jonathan Mar 12 '11 at 15:55

> library(statmod)

> power.fisher.test(.1,.2,200,200,.05)