We collected data regarding the loss of subscriptions for a certain service. We know the losses to be around 1000, and we got responses from around 500 ex-members, explaining the reasons for leaving. The reasons were divided into 6 categories. Is the sample size large enough to extrapolate the sample proportions (ex: reason A/500=5%) to the overall population?
And is there any more resources that inform on this?
Thanks in advance
You can calculate so called simultaneous confidence intervals for multinomial proportions, and see if they are too wide for your purposes.
In R, it can be done like this (data contains the numbers of the respondents from the 6 categories you mentioned):
if(!require(DescTools)){
install.packages("DescTools")
library(DescTools)
}
data <- c(14, 52, 90, 39, 17, 288)
MultinomCI(data)
Output:
A matrix: 6 × 3 of type dbl
est lwr.ci upr.ci
0.028 0.000 0.0719399
0.104 0.062 0.1479399
0.180 0.138 0.2239399
0.078 0.036 0.1219399
0.034 0.000 0.0779399
0.576 0.534 0.6199399
The simplest quick and dirty answer is to quote everything as $\pm 1/\sqrt{N}$, which in this case is 4.5%. This is the same approach used in news media about public opinion polling, in which the standard "plus or minus three percent" means they asked about 1000 people (since $1/\sqrt{1000}\approx 0.0316$). The justification is treating the observed responses like a Poisson process, in which the N obtained is the best estimate of both mean and variance, which would make $1/\sqrt{N}$ the half-width of a one-sigma confidence interval for the rate. It's a popular approach, because it's really easy, but it's nowhere near rigorously true.
What you're really trying to do is estimate a confidence region for a multinomial proportion. However, you can probably get away with ignoring the constraint that all the proportions need to sum to one, and instead separately consider independent confidence intervals for six different binomial proportions. The Wikipedia article is a decent introduction to the the topic; for detailed reference, I recommend Brown, Cai, and Dasgupta (2001), "Interval Estimation for a Binomial Proportion", Statistical Science 16 (2): 101–133, and Newcombe (1998), "Two-sided confidence intervals for the single proportion: comparison of seven methods", Statistics in Medicine 17 (8): 857–872. Of the many approaches described, my favorite is the Wilson score interval, because among the best performers, it's the easiest to explain to non-specialists (it only needs square roots, not the incomplete beta functions of the Jeffreys prior), and it's also recommended by NIST.
To use it, first pick a $z$ score to quantify your confidence in the usual Gaussian way, for example $z$=1.96 for 95% confidence. Then, for each of your six categories $\{X_i\}$, let $p=X_i/N$, and then compute the confidence interval bounds as $$\frac{Np + z^2/2 \pm z \sqrt{Np(1-p)+z^2/4}}{N+z^2}$$where the minus sign gives the lower boundary and the plus sign gives the upper. If you stare at this for a while, you may recognize that you've essentially added $z^2$ coin flips ($\,p$ = $1\!-\!p$ = 1/2) to your data, as pointed out in Agresti and Coull (1998), "Approximate is better than exact for interval estimation of binomial proportions", The American Statistician, 52 (2): 119-126. Note this means your confidence intervals are not symmetric about $p$, which remains the best point estimate of the probability of that response, but that is the small price you pay for the benefit of choosing an interval that never extends past the boundaries (0 and 1).
As the population of interest is of size $\approx 1000$ and your sample half of that population, I do not think that a binomial approximation is warranted here. Instead I would argue for modelling your data using a Hypergeometric (e.g. reason A vs. all others, and cycle through the reasons) or Multivariate hypergeometric distribution (reason A, reason B, etc. all at once).
For the first approach see this SE question including the reference mentioned.
However the problem with response bias mentioned by Henry still stands - any inference will assume that your sample stems from a random draw from that population which is unlikely to be the case. For example if one of the reasons is "the service is sending a lot of annoying emails" then anyone that left due to that reason is unlikely to respond to your survey.
