I'm looking at daily purchasing data over a period of several weeks. Specifically, we're interested in the % of users who make purchases in our app (it doesn't matter the type of purchase or amount spent for our purposes). On any given day, there might be ~10K active users and only about ~30 who actually make a purchase (it's a tough market!). There are three factors that we are interested, call them A, B, and C. Each factor has two levels.
A few rows of data look like:
# NU NP %buy A B C date 1 12407 24 0.0019 0 0 2 2013-03-06 2 11774 37 0.0031 1 0 2 2013-03-05 3 12331 39 0.0032 0 0 2 2013-03-04 4 12679 41 0.0032 1 0 1 2013-03-03 5 12555 61 0.0049 0 1 2 2013-03-02 6 12594 50 0.0040 1 1 2 2013-03-01 7 12466 71 0.0057 0 1 2 2013-02-28 8 12089 61 0.0050 0 0 1 2013-02-27
Here, each row is a daily observation of one group of users with a set of A,B,C factors, NU is the number of total users in that group, NP is the number of users in the group that made a purchase, and %buy is just the % of users in the group who made a purchase. For convenience, let's assume that there are no important time-varying trends in any of the groups, which obviously could complicate the analysis even more (and over a short time period that we're looking at is probably an ok assumption anyway).
My first thought was to do 3-way ANOVA using the %buy as the observational outcome, but then I know I'm neglecting the importance of the sample sizes. Also, I'm thinking that the small proportions might mean a different test is necessary than run-of-the-mill ANOVA. So my question is really what is the proper test that I should be running here to look for significant differences between groups and within groups? Hopefully, I've given enough explanation to make this question clear.