We are running a marketing campaign where we send out comminications to a select group of customers on daily basis. The customers are different each day.
However each day we only have a small number of total customers, average 9 with SD 7.
Each day we randomise the sample into control(10%) and treatment. We round the control group to the nearest integer. For example, if we have 5 customers we allocate 1 to control.
The aim is to look at the behaviour of customers 30 days after email.
To get a sufficient sample size we aggregate samples of many days, so after 10 months we have around 200 control and 1700 treatment.
My question is whether there is any bias or a problem of allocating samples to a control group from such small sample sizes and then adding them all together to do
z-test of proportions?