I am testing the results of a marketing campaign which involves randomly splitting a group of people into 4 treatments. One group acts as a control and is shown no advertising, the other 3 groups are shown one of three different ads. The number of people who bought or didn't buy a product in each group is then calculated.
Performing a chi square test gives me a significant result (dof = 3, p value = 6.5001e-06, chi2 = 26.7948) but I would like to take it a step further and perform a post hoc test similiar to performing a tukey hsd test after ANOVA. I used a simple method and divided the observed value of each cell by the expected value of each cell under independence. The resulting table is shown below:
- Is it fair to make statements like the observed values for those who purchased after seeing test_2 was 1.04 times greater than we would expect under independence? Therefore, test_2 is effective.
- If it is fair to make statements like this, how do I make a real world interpretation of the control group whose observed value is 0.92 times it's expected value?
- The average purchase rate for each group is about 1.2% and the total size of each group is about 220,000.
- I have ran a binomial test in conjunction with this test and the results are similiar.