# Why use G-test and the likes for AB testing at all?

I am learning about AB testing using the G-test. In my example, I have a 2x2 contingency table.

>print(T)
response
AB    no yes Sum
A   29   7  36
B   23  16  39
Sum 52  23  75


Event A is the red background of a website. Event B is the blue background of a website. I showed the website to the total of 75 people. Yes is like, and no is the opposite. After running the G-test I get

>likelihood.test(T)

Log likelihood ratio (G-test) test of independence without
correction

data:  T
Log likelihood ratio statistic (G) = 4.1914, X-squared df = 1,
p-value = 0.04063


The p-value is pretty small (significant at the 5% level), so I reject the null that the samples A and B have the same performance. Now, I have two questions:

1. How do I know what background color is better and has the higher performance?
2. Why do I need the G test at all? I can just compare the percentage of likes for each background color. For red (A), 7/36=19.4%, and, for blue background (B), 16/39=41%. So, clearly B has the higher like percentage, hence, is better. So, why use G test at all?

Remark: I also use the Fisher exact test since one of the measured values, A_yes, is smaller than 10. The output is

> fisher.test(T)

Fisher's Exact Test for Count Data

data:  T
p-value = 0.04948
alternative hypothesis: true odds ratio is not equal to 1
95 percent confidence interval:
0.9167091 9.6380957
sample estimates:
odds ratio
2.841057


The p-value is nearly identical to the G-test's. So again why use the G-test or Fisher test at all when one can just compare the yes percentage for each event. Thanks.

-
Which R packages' likelihood.test is this? The one in Deducer? – Glen_b Jun 15 '14 at 4:16

## 1 Answer

Answering your two questions one at a time (though the answers are related):

1. You only have data on whether they like it, not any information about 'performance'. But the hypothesis test(s) tell you whether or not the difference can be accounted for by chance, and you can tell the direction of the difference in the sample (the thing that would be leading to the rejection) by looking at the data.

2. The problem is whether that apparent sample difference is just due to random variation. Imagine I toss two coins and one came up heads 80% of the time while the other came up heads 60% of the time. Is the first one more likely to come up heads in the long run? If the 80% and 60% was 4 heads and 3 heads in 5 tosses respectively, you could easily see that size of difference just by chance.

So it would be useful to have some idea of whether the difference is more than you could typically explain as nothing but chance. That's not necessarily best dealt with via hypothesis testing (you might look at something like a confidence interval for the difference, which also lets you see whether the size is big enough to regard as of practical importance.

-