# Household Surveys: When is a sub sample 'too small'?

Imagine you have completed a nationally representative survey. You stratified the sample in such a way that you have sufficiently large samples of boys, girls, adult males and adult females. Let us assume a total sample of 2500 individuals.

However, you want to disaggregate the sample further. You want to estimate the percentage of boys/ girls/ adult males/ adult females that - do not know Sesame Street - are aware of Sesame Street - that sometimes watch Sesame Street - that frequently watch Sesame Street

The more you disaggregate the smaller the sub sample size within each sub group. For example out of a sample of 2500 individuals, there might be only 55 men that watch Sesame Street frequently. Let us assume this would be 25%.

The smaller the sample size the more imprecise your sample size. This is due to the confidence interval (CI). The CI is a function of the square root of n. If I had a sample of 550 men that watch Sesame Street frequently the estimate would be 25% +/- 2%, for example, whereas in my current sample of 55 men that watch Sesame Street frequently it might be 25% +/- 8%.

Hence, as long as I also report the confidence interval (and n) of each estimate (e.g. men that watch Sesame Street frequently or girls that are not aware of Sesame Street) I do not run the risk of suggesting higher levels of precision than I actually have. is that correct?

In a similar example, a friend of mine recommended to not even bother looking into men hat watch Sesame Street frequently because n is too small. Therefore the estimate of 25% is practically meaningless. However, is she right? As long as I am aware of CI I can still use this estimate to get an idea of the true population parameter of men that watch Sesame Street frequently.

• You need to review the literature on small area estimation. I'd recommend the book by Rao here: amazon.com/Small-Estimation-Wiley-Survey-Methodology/dp/…. There are many techniques for obtaining reliable estimates within these very small sub-populations. Too many to name here. Feb 18, 2016 at 4:32
• @StatsStudent, thanks so much! I ordered it right away!
– DomB
Feb 18, 2016 at 8:56