# What is the best statistical test to compare the survey Response rate for a control and treatment group?

We run a survey and want to run a controlled test on a sample of the data. The measure we are trying to improve is response rate.

Our research question is:

“does sending out more invites to participate, improve the response rate for the treatment group versus the control group that receive less invites?”

The dependent variable is whether the person takes the survey, and this is a simple categorical variable: respond / do not respond.

However, our measure for this variable is continuous, in that we are measuring the response rate.

We are measuring for both the treatment group and the control group: number of responses/population

Our total population is large 30,000 people split into a 3,000 treatment group and 27,000 people in the control group. Therefore, we think the large population is likely to lead to a statistically significant result even if the difference between the groups could be due to natural variance.

I’m not sure what statistical tests are bests to run on the data? Most tests require a mean and standard deviation, but I’m not sure we can calculate these effectively with a categorical variable. Or whether I should be treating the data as continuous?

I’ve looked at odds ratio test and this seems to be the best fit I can see in terms of comparing the response rates. I don’t know if using Cohens d test might be a better option?

Even if you are most interested in rates, if you have the number of respondents and non-respondents in both groups, the most common test here would be a simple $$\chi^2$$ test on contingency tables.