1
$\begingroup$

Let's say I have a survey with $k$ questions and the sample is representative of the population. So, if I analyse the results for one question, I suppose those results are representative of the population. Let's say I work for a food company. One of the questions is 'Do you like this product?' If the results were 10% yes, 90% no, then I'd feel ok about extrapolating those numbers to the population.

But now I need to drill down a bit. So, among the 90% that answered they didn't like the product, 50% said it was because of the packaging, 20% that it was because of the smell, and 30% that it was because of the taste. Maybe I still feel ok with the extrapolation. But let's say I keep drilling down and down across a number of questions, filtering again and again, and progressively reducing the number of responses in consequence. So my question is, when should I stop thinking that extrapolating the results to the whole population is fine? Is there a mathematical criterion or something? I mean, after I subset the responses $k$ times I wouldn't feel really confident about the representativeness of the results, so I reckon there must be a threshold.

Please go easy on me, I'm a beginner. Thanks.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

The sample should be representative if your sampling scheme is sound.

I believe what you should be concerned about is precision instead. As you drill down on your questions the number of units decreases, then precision decreases. You should sample a number of people that is enough to obtain a low variance for the estimator in the most detailed case.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Ok, so If I got it right, then a sample can give accurate results that are imprecise if it is not large enough. Is there a special formula to calculate the ideal sample size that takes into account the maximum level of drill down? $\endgroup$
    – numberfive
    Commented Sep 24, 2020 at 18:07
  • $\begingroup$ There are formulas to estimate the sample size given a precision threshold. They depend on the sampling design and the estimator of interest and can be very complex depending on the scenario. In your case, you might want to look for the sample size for a proportion, given a stratified sampling, I guess given your description. $\endgroup$
    – Roberto
    Commented Sep 25, 2020 at 7:54
  • $\begingroup$ Thanks, Roberto. $\endgroup$
    – numberfive
    Commented Sep 26, 2020 at 13:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.