How come Z-test is recommended by some for analyzing A/B tests? I've been researching the deep underlying mathematics of A/B testing. I understand A/B tests to be Bernoulli trials (conversion / non conversion) and categorical data represented in a 2x2 contingency matrix.
Why and how is it that many blogs or websites recommend using a Z-test to measure whether the two sample frequencies are drawn from the same population (accept null hypothesis), or are significantly different meaning they're drawn from different populations (reject null hypothesis)? How can a Z-test even be applied here?
This Wikipedia page doesn't list Z-test for categorical data: http://en.wikipedia.org/wiki/List_of_analyses_of_categorical_data
I understand that a G-test or Chi-square test is suitable, but does not provide a confidence interval, only a p-value to compare to a significance level. Is this related?
 A: 
Why and how is it that many blogs or websites recommend using a Z-test to measure whether the two sample frequencies are drawn from the same population (accept null hypothesis), or are significantly different meaning they're drawn from different populations (reject null hypothesis)? How can a Z-test even be applied here?

As a two-sample proportions test.
One advantage over a chi-square is you can do a one-tailed test.

This Wikipedia page doesn't list Z-test for categorical data: http://en.wikipedia.org/wiki/List_of_analyses_of_categorical_data

This one does, search for "Two-proportion z-test".
A: The chi-squared of a 2x2 A/B test has only (2-1)x(2-1) = 1 degree of freedom, so it's equivalent to a Z-test. (A chi-squared distribution comes from the sum of squared z-scores.) In this specific case, the critical value of the Z-test is the square root of the critical value of the chi-squared test.
A: I had the same confusion. A good example is here: https://www.khanacademy.org/math/ap-statistics/two-sample-inference/two-sample-z-test-proportions/v/hypothesis-test-for-difference-in-proportions-example
You can just replace the situation with conversion rates.
