I am setting up an A/B test of a webpage where users will be randomly distributed between either A or B. I'll be using the N-1 Proportion Test to determine which page converts better than the other.

I'd like to look the traffic going to each treatment and confirm the samples are similar in terms of referrers. I have the proportion of traffic coming from Organic Search, Direct, Email campaigns (6 inbound channels total).

Since the sizes of the two samples are different, I worry Chi-Squared Goodness of Fit won't be the best test. I can do a N-1 Proportion test on each of the ratios (so, A's Organic percentage in relation to B's Organic Percentage), but I'm not sure if that is the right test.

Ultimately, I'd like to see if the proportions are all about the same (or at least not out of reason). Is it best to run the proportion test on each individually or run something else?

So, I have two sample groups each that can be segmented in six groups. What's the best test?


Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.