This tool assumes a normal distribution and uses difference of means to compute confidence.
What is the difference between a G test and a T test? What are the benefits or downsides to using each method to measure the effectiveness of our A/B tests?
I'm trying to figure out which one I should use to measure the results of my A/B test framework. Our framework has two general use cases: split the group of visitors evenly, show each one a different feature and measure their conversion on some other page (say, the sign up page); and split the group of visitors into the control group (90%) and an experimental group (10%) for a test, and measure conversions on some other page.
Our website gets between 1000 and 200,000 visits per day (I am being vague on purpose to hide the true number, which doesn't change much). These visits are split with an exponential distribution across about 300 pages.