Consider a population characterized in terms of a small number of categorical features (age category, income bracket, gender, level of education, etc.). I am interested in how these features affect the probability that an individual will responds to an online advertisement. Assume that I have the ability to sample individuals with any set of characteristics from this population (i.e., show them the ad and see how they respond).
What is the standard procedure for using design of experiments to build a model that tells me which features are relevant? Is it simply a full/partial factorial design where I show the ad to all possible combinations of feature values, and use ANOVA to show me main effects and interaction effects?
Or, is ANOVA invalidated by the fact that my response variable is binary? In that case, should I rather sample from the population to "generate" the response variable, and then build a logistic regression model? Do I miss out on interaction effects by doing that?
Finally, what are the main "tradeoffs" one faces in this problem? Is it only that sampling data points is expensive, but in theory you get more precise results?