Sample size choice with binary outcome

Our civic association is planning to implement a survey asking our neighborhood one true false question. I would like to make sure that we obtain an adequate sample size.

Let's assume N = 2500 people, and we want to be 90% confident of our results with margin of error no more than 5%.

We would like to know the minimum sample size. Is there a simple formula to apply? If so can you please share it?

If so, is there an easy way to stratify the sample by some categorical variable such as race. For this example let's assume 25% of the neighborhood is part of a minority group. What is the minimum number of minority households that must be included?

Including a finite correction factor, the margin of error $m$ for binary data is given by $$m = z_{\alpha/2} \sqrt{{p(1-p)} \over {n}} \sqrt{{{N-n} \over {N-1}}},$$ where $N$ is the population size, $n$ is the sample size, $p$ is the success probability (say, the chance of a voter giving a true answer), and $z_{\alpha/2}$ is the appropriate quantile from the standard normal distribution for the desired confidence.

Solving for $n$ we find $$n ={ {z^2_{\alpha/2} \ p (1-p)N} \over {z^2_{\alpha/2} \ p(1-p)+(N-1)m^2} }$$ Now $p$ is unknown, but the worst case is $p={{1} \over {2}},$ so using that we have $$n ={ {z^2_{\alpha/2} \ N} \over {z^2_{\alpha/2} \ +4(N-1)m^2} }$$

Using your numbers ($z_{\alpha/2}=1.6449$), I get a sample size of $n=244.2,$ so use $n=245.$

Ignoring the finite correction factor would have led to a sample size of $n=272.$

For your second question, I'm not sure what you are asking. The minimum number of minority households to sample may be more of a political issue than statistical.

• I am not getting the same result: (1.6449*2500)/(1.6449+4*2499*0.05^2) =154.3933. Clearly I am missing something. Can you see what I did wrong? – mmann1123 Apr 18 '16 at 21:13
• Make sure you square the $z$ factor. – soakley Apr 18 '16 at 21:14
• With the minority question, Soakley may be alluding to that different groups may have different survey response rates? Eg. if old people oppose some measure (eg. school funding) and respond to surveys at higher rates than young people, you may underestimate support for school funding in the population by looking at survey responses. Anyway, you can get into extremely tricky issues if various covariates (eg. age) are linked with both public opinion and survey response rates. – Matthew Gunn Apr 18 '16 at 22:37
• To get the desired accuracy on both strata will require much more sampling. With 1875 nonminority households you will need 237 data points, and with the 625 minority households you will need to sample 190. If you have reason to believe the true success probability is much different than 0.5, then you can improve on these using the first formula for $n$ in the answer. – soakley Apr 19 '16 at 17:38