I am learning about AB testing using the G-test. In my example, I have a 2x2 contingency table.
>print(T) response AB no yes Sum A 29 7 36 B 23 16 39 Sum 52 23 75
Event A is the red background of a website. Event B is the blue background of a website. I showed the website to the total of 75 people. Yes is like, and no is the opposite. After running the G-test I get
>likelihood.test(T) Log likelihood ratio (G-test) test of independence without correction data: T Log likelihood ratio statistic (G) = 4.1914, X-squared df = 1, p-value = 0.04063
The p-value is pretty small (significant at the 5% level), so I reject the null that the samples A and B have the same performance. Now, I have two questions:
- How do I know what background color is better and has the higher performance?
- Why do I need the G test at all? I can just compare the percentage of likes for each background color. For red (A), 7/36=19.4%, and, for blue background (B), 16/39=41%. So, clearly B has the higher like percentage, hence, is better. So, why use G test at all?
Remark: I also use the Fisher exact test since one of the measured values, A_yes, is smaller than 10. The output is
> fisher.test(T) Fisher's Exact Test for Count Data data: T p-value = 0.04948 alternative hypothesis: true odds ratio is not equal to 1 95 percent confidence interval: 0.9167091 9.6380957 sample estimates: odds ratio 2.841057
The p-value is nearly identical to the G-test's. So again why use the G-test or Fisher test at all when one can just compare the yes percentage for each event. Thanks.