1
$\begingroup$

Imagine you have completed a nationally representative survey. You stratified the sample in such a way that you have sufficiently large samples of boys, girls, adult males and adult females. Let us assume a total sample of 2500 individuals.

However, you want to disaggregate the sample further. You want to estimate the percentage of boys/ girls/ adult males/ adult females that - do not know Sesame Street - are aware of Sesame Street - that sometimes watch Sesame Street - that frequently watch Sesame Street

The more you disaggregate the smaller the sub sample size within each sub group. For example out of a sample of 2500 individuals, there might be only 55 men that watch Sesame Street frequently. Let us assume this would be 25%.

The smaller the sample size the more imprecise your sample size. This is due to the confidence interval (CI). The CI is a function of the square root of n. If I had a sample of 550 men that watch Sesame Street frequently the estimate would be 25% +/- 2%, for example, whereas in my current sample of 55 men that watch Sesame Street frequently it might be 25% +/- 8%.

Hence, as long as I also report the confidence interval (and n) of each estimate (e.g. men that watch Sesame Street frequently or girls that are not aware of Sesame Street) I do not run the risk of suggesting higher levels of precision than I actually have. is that correct?

In a similar example, a friend of mine recommended to not even bother looking into men hat watch Sesame Street frequently because n is too small. Therefore the estimate of 25% is practically meaningless. However, is she right? As long as I am aware of CI I can still use this estimate to get an idea of the true population parameter of men that watch Sesame Street frequently.

Improve this question
$\endgroup$
2
  • $\begingroup$ You need to review the literature on small area estimation. I'd recommend the book by Rao here: amazon.com/Small-Estimation-Wiley-Survey-Methodology/dp/…. There are many techniques for obtaining reliable estimates within these very small sub-populations. Too many to name here. $\endgroup$
    – StatsStudent
    Feb 18, 2016 at 4:32
  • 1
    $\begingroup$ @StatsStudent, thanks so much! I ordered it right away! $\endgroup$
    – DomB
    Feb 18, 2016 at 8:56

1 Answer 1

Reset to default
0
$\begingroup$

no. assumptions break down at very small sample sizes. use an exact test if you need to present a finding regardless of the width of its confidence interval. even then, you are better off presenting the confidence interval rather than the statistic. it's prolly misleading to say that a number is 25% when in reality all that you know is that the number is somewhere between 3% and 55%. safer to show the range

Improve this answer
$\endgroup$
2
  • $\begingroup$ ! Thanks for the quick reply! My intention is to present the statistic as well as the CI. Would you say it is even standard to just present the range in such cases? $\endgroup$
    – DomB
    Feb 18, 2016 at 1:44
  • $\begingroup$ hi, when the confidence interval is absurdly wide, it is more honest to present the range than to present the statistic. $\endgroup$
    – Anthony Damico
    Feb 18, 2016 at 4:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.