# Tagged Questions

91 views

### Understanding Sequential Probability Ratio Test (SPRT) Likelihood Ratio

I am a software developer looking to develop an alternative for the simple hypothesis testing scheme described here. In short, the test works as follows: Two URLs are compared for their ability to ...
48 views

### which test to use if you only have summary statistics? (non-normal)

Here's the basic problem: visiters to our website were randomly put into one of 7 different groups, and we want to compare the activity of users in the different groups on a downstream site. The ...
111 views

### How to interpret type-II error probability while doing A/B testing?

I've been trying to familiarize myself with some of the statistics behind conducting A/B testing on a website and trying to pick a relevant sample size. I've come across several articles that talk ...
349 views

### Analyze results from a/b test using welch's t test

I'm performing a web based a/b test where there is a control and one treatment. The results are not as simple as "converted" or "didn't convert." A user can "convert" anywhere between 0-10 times. I ...
785 views

### Safely determining sample size for A/B testing

I am a software engineer looking to build an A/B testing tool. I don't have a solid stats background but have been doing quite a bit of reading over the last few days. I am following the methodology ...
182 views

### A/B Testing of a probability, interpreting results of online conversion test

So I have a formula to calculate sample size for a Bernoulli trial, given the inputs of the Delta I wish to detect, and the current probability of a success (click-through rate or CTR). ...
841 views

### Bayesian AB testing

I am running an AB Test on a page that receives only 5k visits per month. It would take too long to reach traffic levels necessary to measure a +-1% difference between the test and control. I have ...
294 views

### How to calculate sample size for a one-sided test on a rxs contigency table

I work quite often with Google's Website Optimizer, essentially it allows you to make small changes to a site to determine if they have an effect on the ratio of clicks to a web page to sales. The ...