Skip to main content
5 of 6
added 154 characters in body
Parseval
  • 363
  • 3
  • 9

Statistical test for 3 groups (A/B/C test) for binary outcome. (binary dependent variable)

We send out daily e-mails to customers suggesting products at different times: 09:30, 12:00, 19:30. A customer can either click on a product or not. I want to know the following: Is there a significant difference in clicks depending on at what time an email is sent to the customer?

The hypothesis is set up as follows:

\begin{align} \mathcal{H}_N &= \textrm{There is no difference in number of clicks between time groups} \\ \mathcal{H}_A &= \textrm{There is a difference in number of clicks between time groups} \end{align}

The data set I have is the following

> summary(df)
 Click      Time      
 0:277551   0930:93799  
 1:3236     1200:93446  
            1930:93542  

Where 0=no click and 1=click. My first guess was a one-way ANOVA but then I have to make the assumption that my dependent variable Click is continous and normally distributed, which is not the case.

What would be an appropriate test for the scenario I've described? If I only had two timegroups I'd use test of two proportions as suggested here. Is there any test of 3 proportions?

EDIT 1: Data set as per Ben Bolkers suggestion. But here I have only 3 rows and not 6 as he suggests. I'm misunderstanding what he means.

enter image description here

EDIT 2: Fitting glm as dipetkov suggested gives the following result, using the raw data set in the form

Click Time
-----------
0     0930
1     0930
1     1200
0     0930
0     1930
...

Call:
glm(formula = Click ~ Time - 1, family = binomial, data = df)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-0.1595  -0.1595  -0.1520  -0.1450   3.0200  

Coefficients:
         Estimate Std. Error z value Pr(>|z|)    
Time0930 -4.54982    0.03210  -141.8   <2e-16 ***
Time1200 -4.45538    0.03070  -145.1   <2e-16 ***
Time1930 -4.35849    0.02927  -148.9   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 389253  on 280787  degrees of freedom
Residual deviance:  35301  on 280784  degrees of freedom
AIC: 35307

Number of Fisher Scoring iterations: 7

All the groups seem to be significant. How do I find which one of them leads to most clicks?

Parseval
  • 363
  • 3
  • 9