4
$\begingroup$

I have a population of phone calls - 200,000. There are different reasons for each call, but lets assume the number of reasons is known. i.e. 7 different call reasons: 1) Check on order 2) Cancel order 3) Billing information 4) Account information etc...

My question is, I want to get a statistically significant sample size so the number of calls I listen to in my sample are representative of the distribution of the reasons in the overall population of 200,000 calls.

What should the sample size be? Which methods to use to calculate the sample size?

$\endgroup$
6
  • 4
    $\begingroup$ What does "representative of ..." mean in this context? $\endgroup$ Commented Mar 17, 2015 at 18:29
  • $\begingroup$ To clarify: Do you know the reason(s) for each call in advance, or is your purpose to listen and ascertain the the reasons? Are the calls ordered in anyway (e.g. in time)? Do you have any other advance information about them, e.g. length of call, that might be important? $\endgroup$ Commented Mar 17, 2015 at 22:20
  • $\begingroup$ @gung - representative means being confident that the sample size I have taken will include all the reasons that customers could be calling about and in the right proportions. i.e. IF the original population the distribution was Check order-20% Cancel-15% Billing-10%, etc... Then my sample will be ave the same distribution. $\endgroup$
    – Zorro
    Commented Mar 19, 2015 at 11:29
  • $\begingroup$ @SteveSamuels - I don't know the reasons before hand for each call, the purpose is to ascertain the reasons. The calls do have a timestamp, and there are other information associated with the length of the call, but lets assume these factors don't make a difference. i.e. calls for cancels and billing are randomly distributed through time and that the length of each call is not linked to the length of the call. $\endgroup$
    – Zorro
    Commented Mar 19, 2015 at 11:34
  • 1
    $\begingroup$ If you need to ensure that you have, say, 20% check orders, then you need to sample, say, 20 calls at random from w/i those that were check orders, & 15 calls from w/i the cancels, etc. If you can't do that, then you need to use all calls to ensure you get those numbers. $\endgroup$ Commented Mar 19, 2015 at 15:12

1 Answer 1

8
$\begingroup$

In the absence of auxiliary information, I recommend simple random sampling. It is safe to ignore the fact that you are sampling from a finite population, since the population size $N = 200,000$ is huge, compared to the sample size $n$. Essentially we assume that draws are independent.

Each call can be classified into k = 7 categories. With the simple random sampling design, the number of appearances $a_1, a_2, \ldots a_k$ follow a multinomial distribution, with true proportions $\pi_j$, $j = 1\ldots k$. These will be estimated by the sample proportions $p_j$.

Thompson, 1987, studied sample size determination for estimating a simultaneous $1-\alpha$ confidence set for the $P_j$ with multinomial data. The individual intervals each have the same half-length $2 d$. That is, each interval is of the form: $$ \left[\,p_j -d,\, p_j +d\,\right] $$

The analyst chooses $d$ and a protection level $\alpha$, which will be the maximum probability that one or more of the $k$ intervals does not contain its corresponding population proportion; i.e.:

$$ \text{Pr}(|\,P_j - p_j\,| > d\text{, for any j}) \le \alpha. $$

Equivalently, the probability that all the intervals cover the true $P_j$ is greater than $1-\alpha$.

Thompson proved that the worst case configuration of the $P_j$ in this setup, the one requiring maximum $n$, is one in which the true proportions are equal: $P_j = 1/k$. He provided a simple table for choosing $n$, or, if resources limit $n$, for balancing the choices of $\alpha$ and $d$. These results apply as long as $k$ exceeds a certain minimum value that varied with $\alpha$.

 alpha  (d^2 x n)    min k    n if d = 0.05
 ----------------------------------------
.50      .44129        4        177  
.40      .50729        4        203
.30      .60123        3        241
.20      .74739        3        299
.10     1.00635        3        403
.05     1.27359        3        510
.025    1.55963        2        624
.02     1.65872        2        664
.01     1.96986        2        788
.005    2.28514        2        915
.001    3.02892        2       1212
.0005   3.33530        2       1342
.0001   4.11209        2       1645

You can see that $k=7$ is covered by the table, since 7>4. For each $\alpha$ under consideration, choose $d$, then solve for $n$ by dividing the value in column 2 by $d^2$. Alternatively,if your resources or time limit you to a maximum sample size $n_{\text{max}}$, use column 2 solve for $d$. If you were to choose $\alpha =0.20$ and $d = 0.05$, for example, the corresponding value of $n = 0.74739/0.05^2 = 298.954$, rounded up to 299 in column 4.

Other methods for computing simultaneous confidence intervals

Sison and Glaz (1995) proposed two methods for finding simultaneous confidence intervals and recommended their Method 1. May and Williams (2000) published a SAS macro to calculate a version of Sison and Glaz's Method 1. Hou et al. (2003) presented still another method.

References

Hou, Chia-Ding, Jengtung Chiang, and John Jen Tai. 2003. A family of simultaneous confidence intervals for multinomial proportions. Computational Statistics & Data Analysis 43, no. 1: 29-45.

May, Warren L, and William D Johnson. 2000. Constructing two-sided simultaneous confidence intervals for multinomial proportions for small counts in a large number of cells. Journal of Statistical Software 5, no. 6: 1-24. preprint: http://www.jstatsoft.org/v05/i06/paper

Sison, Cristina P, and Joseph Glaz. 1995. Simultaneous confidence intervals and sample size determination for multinomial proportions. Journal of the American Statistical Association 90, no. 429: 366-369. http://140.112.142.232/~purplewoo/Literature/!Methodology/!Distribution_SampleSize/SimultConfidIntervJASA.pdf

Thompson, Steven K. 1987. Sample size for estimating multinomial proportions. The American Statistician 41, no. 1: 42-46.

$\endgroup$
11
  • $\begingroup$ Nice answer (+1). I suspect that your suggestions are somehow related to distribution-independent standard approach of Cohen (1988). It would be interesting to know, what is the connection (references), if any, as well as why your approach is preferable to the standard one (based on statistical significance, population effect size and statistical power). $\endgroup$ Commented Mar 24, 2015 at 23:25
  • 2
    $\begingroup$ Thanks for the vote. The OP does not say anything about testing hypotheses. He wants to know how big a sample will be "representative"; which, after soliciting more information, I interpreted as having sample proportions close to population proportions. So it's an estimation problem. There's no connection to the literature of hypothesis testing, effect size, power, and statistical significance. I say this, even though you can always invert individual confidence intervals to perform point hypothesis tests; you can, but it's not an issue here. $\endgroup$ Commented Mar 25, 2015 at 21:22
  • 2
    $\begingroup$ I should add that the literature of multiple comparisons (e.g. JC Hsu (1996) Multiple Comparisons: Theory and Methods, Chapman and Hall, which happens to be on my bookshelf) often has a section on simultaneous confidence intervals for single parameters (Hsu, Chapter 1) before proceeding to the "Comparisons" topics. $\endgroup$ Commented Mar 25, 2015 at 21:56
  • 1
    $\begingroup$ I'm perplexed. If Sison and Glaz is so good at handling the case of all equal true proportions and Thompson says that this condition will have the worst case largest required sample size, then why not use the Sison based method for estimating required sample size? Yet, May and Johnson say that these confidence intervals are not great in other situations. But couldn't you just use the other confidence procedures once you have the data and know what to expect? $\endgroup$
    – wdkrnls
    Commented Sep 16, 2022 at 17:44
  • 1
    $\begingroup$ @wdkrnls I have the same questions. In case you didn't find it and are still interested, the Thompson's paper is available for download here: jstor.org/stable/2684318 $\endgroup$
    – J-J-J
    Commented May 31, 2023 at 7:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.