A/B testing where success is not binary I am a blogger testing different ad networks. The problem statement is actually quite simple:
Ad network 1: 100,000 pageviews. $460 ad revenue  
Ad network 2: 100,000 pageviews. $428 ad revenue
Can I say with 95% certainty that ad network 1 is better than ad network 2? 
The sub-questions are as follows:


*

*Am I asking the right question here? Or should I be asking "can I say with 95% certainty that ad network 1 is 10% better than ad network 2?"

*In this scenario, is it appropriate to use a chi-squared test counting each dollar earned as a success? If we did that, it would look like this. So in dollars, there is no statistically significant difference. But if we change to each cent earned as a "success", then there is a significant difference. So this method doesn't seem to pass the smell test.

*The test that answers the question "Does the average value differ across two groups?" is the two-sample T-test. But I don't have individual data points here; just aggregates. Is there an alternative to the T-test that can be used for aggregate data?

 A: You have actually done most of the hard work with this problem already. The trick is to think about it in terms of a t-test.  When you compute the t-test, the inputs are: the means, the sample sizes and the standard deviations.  In your case, the means are fixed (0.0046 and 0.0048 per click), the sample size is fixed (100,000 and 100,000). You are finding no significant difference when you assume \$1 a click, but you are finding a significant difference when you assume \$0.01 per click.  This is because, for example, with your network 1, \$1 equates to a standard deviation of 0.06766746 where \$0.01 is 0.04983999, and the bigger the standard deviation the bigger the p-value (which is the complement of what you refer to as "certainty", which, sadly, is not really certainty, but read up on frequentest statistics if you wish to go down that rabbit hole).
Without knowing the standard deviations there is no way to test for your statistical significance, as your two alternative sets of assumptions demonstrate. But, what you can do is find some other bit of data that informs you about the likely ad revenue per click, as this will allow you to work out the range of possible standard deviations. For example, if the revenue is coming from store purchases and the cheapest thing in the store is \$1, then you have already worked out that there is no significant difference between the networks.
