Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I would like to calculate my ideal sample using the Qualtrics calculator:
https://www.qualtrics.com/de/erlebnismanagement/marktforschung/stichprobenrechner/
For this, I need a confidence level and a margin of error. Do the confidence interval and the margin of error together have to add up to 100%?
In order to find the required sample size $n,$ you need a confidence level (such as $.95 = 95\%)$ and a margin of error (such as $\pm .03 = \pm 3\%).$
So that an explicit answer to your question doesn't get lost in a longer explanation of confidence intervals: No. There's no restriction that the confidence level and the margin of error must add to $1.$
The calculator in the link also asks for a population size, but that is not important unless you're thinking you might sample more than 10% of the population. So if this is for a nationwide poll in a large country with millions of eligible subjects, you can ignore that part. (If you're using the calculator in the link, you'd enter something like $10\,000\,000).$
The margin of error for a 95% confidence interval from a poll is $\pm 1.96\sqrt{\frac{p(1-p)}{n}},$ where $n$ is the sample size and $p$ is the true population proportion with the relevant attribute (such as favoring Proposition A on in an upcoming election).
The margin of error is the proportion (percentage in your link) that determines the width of your confidence interval. Maybe you'd like to say that the true proportion is $0.55 \pm 0.03$ or $55\% \pm 3\%.$ Then $E = .03 = 3\%.$
Not knowing $p,$ you could either guess what $p$ might be, or take the worst case, which is $p = 1/2$ (giving the largest possible margin of error). Then for a 95% confidence interval (CI), you'd have a CI of the form $\hat p \pm E.$ So $E=1.96\sqrt{\frac{p(1-p)}{n}}.$ If you're taking $p = 1/2,$ then you have $E = 1.96\sqrt{.25/n} \approx 1/\sqrt{n}.$ So, if $E = 3\%,$ then $n \approx 1/(.03^2) = 1111$ subjects.
Note: Here's why I say that $p = 1/2$ is the 'worst case', leading to the largest margin of error. The factor $Q = p(1 - p)$ in the margin of error reaches its maximum when $p = 1/2.$ So the margin of error $E$ is maximized when $p = 1/2$ and for a fixed value of $E$ that leads to the largest required $n.$
plot(p, Q, type="l", lwd=2)
abline(v = 1/2, col="green2")
