Chi Square Sample Size Needed with G*Power

I was recommended to try G*Power so I could determine the sample size needed in order to achieve an acceptable power. I'm struggling with picking the test and filling out the appropriate values.

Basically, I'm sending a survey to approximately 3,000 people and am testing different times/days for response rate. So, for example, I'd like to send out on Monday, Wednesday, Friday, and Sunday, in the early morning, afternoon, evening, and middle of the night. So a total of 16 groups (4 x 4 conditions). This would give me 187.5 people in each group. Can anyone explain how I use G*Power or other software to figure this out?

When I select $X^2$ (chi-square) and goodness-of-fit and then "A priori" to calculate the required sample size, it asks me to determine p(H1) for each group, and I really don't know what to expect groups to achieve in terms of response rate. Furthermore, is a chi-square the right test, or should I be using an ANOVA (I am under the impression that ANOVA is for continuous variables, and I'd have a proportion)?

I'm going to steer you away from power calculations so that you can reconsider your procedure. Your situation involves 2 independent variables--day and time--, so you need to replace Chi-square with a 2-factor, multivariate model. Which type you use depends on how you operationalize your dependent variable (DV).

Rather than response rate, which applies to groups, your DV, if measured at the individual level, would be response/nonresponse. And to test for days and times that relate to this it would be natural to use logistic regression. As part of this you could build in a term to test for a day*time interaction, i.e., for specific combinations of days and times that have especially high or low response.

If you prefer or need to measure response at the group level via response rate, then I'm guessing loglinear modeling would be your choice.

• Is there an advantage of operationalizing at the individual level, as you suggest, compared to at the group level (response rate)? Would my conclusions be different? And would an ANOVA be an acceptable test if I use response rate at the group level? Also, what do you mean by "build in a term to test for a day*time interaction"?
– Eric
Mar 9, 2017 at 2:33