I'm programatically adding keywords to bid on. Some keywords will trigger ads and impressions, and some impressions will trigger clicks. Clicks / impressions = CTR (click-through-rate). Clicks cost. Impressions are free. There will be millions of keywords so even if the CTR is low it won't matter as long as the keyword generates sales/revenues/profits.
I'm trying to find how many impressions a keyword has before it is statistically significant. I thought I would use a Z-test, but there is no baseline to measure against. Especially for a new account. Even if the CTR is 0.01%, it doesn't matter because we have millions of keywords, so it doesn't hurt as long as it generates revenue. It could be 1 cent per month profit and still be worthwhile. Then I read someone mention an F-test. But I think that measures against other populations. So I would have to have an account baseline first to compare against.
Basically if a keyword generates revenue, and the profit is greater than the cost of that keyword, the CTR or CPC or CPA is irrelevant.
For example, let's say we have two keywords, 'iphone' and 'android' and 'blackberry'. iPhone generates 1000 impressions, 500 clicks, costs \$50, and generates \$75 revenue. This would be a good keyword, about 50% CTR and positive profit.
Android generates 1000 impressions, but only 10 clicks, costs \$1, and generates \$1.50 in revenue.
Blackberry generates 100 impressions, 1 click, costs $0.10, and no revenue.
How do I know how many impressions or clicks from Blackberry do I accept before declaring it a total loss with a 95% confidence interval? 200 impressions? 10 clicks? There will be millions of keywords, but not unlimited money, so I cannot allow bad keywords to linger and rack up costs. However, I don't want to prematurely end them as well.
Please provide the actual formula or the link or reference.