6
$\begingroup$

I have N different possible results (ex. red, yellow, or blue). I go to a population of infinite size and ask a bunch of people a question the answer to which is one of the N options.... How large will my sample size need to be to have a ~95% confidence that the sample's distribution of answers represents the full population's?

For example, with red, yellow, and blue, how many people do I need to ask the question before I can have a good sense of what %s each answer would get among the full population (within a + or -5% margin of error)? Those numbers are less important to me; what matters more is the mathematical logic of how to even approach this problem.

I have looked for formulas to determine sample size, but all of them take in a population variable and do not ask for the number of possible results. Here, my concern is the number of possible results, and the population is infinite.

Thanks!

$\endgroup$
6
  • $\begingroup$ crossposted: math.stackexchange.com/q/86802/18830 $\endgroup$
    – varty
    Commented Nov 29, 2011 at 19:38
  • $\begingroup$ To make any progress, Carl, you need to quantify what you mean by "good sense of." You also need to be more specific about what it would mean for the sample distribution to "represent" that of the full population. What kinds of discrepancies are ok, and how large can they be? Even after supplying this information, you will need to be content with either a provisional or a worst-case answer (that is, an overestimate of the necessary sample size), because--as you indicate--the optimal sample size depends partly on the population characteristics themselves. $\endgroup$
    – whuber
    Commented Nov 29, 2011 at 19:45
  • 2
    $\begingroup$ BTW, cross-posting is discouraged for many reasons. Please decide where you want this question to appear first and delete the other copies. (IMHO it belongs here: you'll likely get theoretical mathematical answers on the Math site and you're more likely to get practical and nuanced answers here.) $\endgroup$
    – whuber
    Commented Nov 29, 2011 at 19:47
  • 1
    $\begingroup$ Thanks @whuber, I deleted the other post. However, I think it falls under both math and stats, and I am not sure why I should be discouraged from engaging different relevant communities in a discussion that would interest them. $\endgroup$ Commented Nov 29, 2011 at 21:52
  • 1
    $\begingroup$ Concerning cross-posting, please see this thread, Carl. $\endgroup$
    – whuber
    Commented Nov 29, 2011 at 21:56

2 Answers 2

6
$\begingroup$

You should consider modeling the situation using the multinomial distribution. I am going to change variables as I would prefer to reserve $n$ for sample size and denote the number of choices by $K$ (i.e., $K$ represents the number of colors, answers etc).

Let $p_k$ be the true proportion of people in the population who would choose the $k^\text{th}$ choice when presented with $K$ choices. You can re-interpret $p_k$ as the probability that a random person would choose the $k^\text{th}$ choice when presented with $K$ choices. Thus, by definition, we have:

$$\sum_{k=1}^K p_k = 1$$

Let $x_k$ stand for the number of people who choose the $k^\text{th}$ object when we sample the choices of $n$ people. Then the density function of ${x_k}$ is given by the multinomial pdf:

$$f(x_1,...x_K|-) = \begin{cases} \frac{n!}{x_1! ... x_K!} p_1^{x_1} ... p_K^{x_K} \quad \text{if} \quad \sum_kx_k=n \\ 0 \quad \text{otherwise}\end{cases}$$

You can then use maximum likelihood theory to estimate $\{p_1,p_2,...p_K\}$ and compute confidence intervals for these estimates.

Computing the confidence intervals would also enable you to compute margin of errors associated with your estimates for a given sample size. These margin of errors will help you calculate the necessary sample sizes to attain a margin of error of 5% with 95% confidence.


MLE, Margin of error and Sample Size Computations

It is not difficult to show that the MLE estimate for $p_k$ is given by:

$$\hat{p}_k = \frac{x_k}{n}$$

The above estimator is unbiased as:

$$E(\hat{p}_k) = \frac{E(x_k)}{n} = \frac{n p_k}{n}=p_k$$

The variance of the estimator is:

$$V(\hat{p}_k) = \frac{V(x_k)}{n^2} = \frac{n p_k (1-p_k)}{n^2}=\frac{p_k (1-p_k)}{n}$$

Assuming that $n$ is sufficiently high, we can use the central limit theorem to approximate the distribution of $\hat{p}_k$ as a normal with the mean at $p_k$ and with variance $\frac{p_k (1-p_k)}{n}$.

Thus, the margin of error is given for a 95% confidence interval is given by:

$$1.96 \sqrt{\frac{p_k (1-p_k)}{n}}$$

We do not know $p_k$ apriori. However, a conservative estimate for $p_k$ would be that it equals $K^{-1}$ (i.e., we assume that all choices are equally likely). The above argument is a bit ad-hoc but perhaps serves the OP's purpose.

Therefore, we have the requirement that:

$$1.96 \sqrt{\frac{K^{-1} (1-K^{-1})}{n}} = 0.05$$

If we let $K=3$ we get the required sample size as $n=341.475$.

PS: The past question on Asymptotic distribution of multinomial seems relevant in the above context and may suggest ways to lend rigor to the above ideas.

$\endgroup$
4
  • $\begingroup$ I think the question is asking something different, varty. If you want every margin of error to be within 5% with 95% confidence, how large should the sample be and how should it be obtained? $\endgroup$
    – whuber
    Commented Nov 30, 2011 at 2:43
  • $\begingroup$ @whuber Oh, I know. At the very least the above approach will get the OP started with something. One can compute margin of errors associated with the probability estimates and use that to compute required sample sizes. For $K=2$ the above collapses to the well known polling problem for which sample sizes are computed using margin of error. I am a bit lazy to actually look at MLE estimates and associated margin of errors! $\endgroup$
    – varty
    Commented Nov 30, 2011 at 2:49
  • $\begingroup$ I don't understand how you got your 1.96 figure for the 95% confidence interval. $\endgroup$ Commented Nov 30, 2011 at 21:07
  • $\begingroup$ @CarlBenson For a normal distribution the z-value associated with a 95% confidence interval is 1.96. See the wiki link (en.wikipedia.org/wiki/1.96) for an explanation. Let me know if the link does not help and I will add further details to my answer. $\endgroup$
    – varty
    Commented Nov 30, 2011 at 21:10
0
$\begingroup$

How is this different than asking independent questions on a survey such as who will you vote for - red, yellow, etc. If I've read the question correct, your answer is 364 observations. This will provide +/-5% at the 95% level of confidence given a normal distribution.
Dr. Doug

$\endgroup$
2
  • $\begingroup$ How did you get to 364? $\endgroup$
    – varty
    Commented Nov 30, 2011 at 15:49
  • 1
    $\begingroup$ You are correct. Could you explain the "logic of...how to approach this problem" requested in the question? $\endgroup$
    – whuber
    Commented Nov 30, 2011 at 15:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.