When learning the course of sampling, I meet the following two statements:

1) Sampling error leads to mostly variability, nonsampling errors lead to bias.

2) Because of nonsampling error, a sample is often more accurate than a CENSUS.

I do not know how to understand these two statements. What is the underlying logic for getting these two statements?

  • 5
    A true census or an attempted one? – cardinal Jan 20 '12 at 17:38
up vote 16 down vote accepted

A sample could be more accurate than a (attempted) census if the fact of the exercise being a census increases the bias from non-sampling error. This could come about, for example, if the census generates an adverse political campaign advocating non-response (something less likely to happen to a sample). Unless this happens, I can't see why a sample would be expected to have less nonsampling error than a census; and by definition it will have more sampling error. So apart from quite unusual circumstances I would say a census is going to be more accurate than a sample.

Consider a common source of nonsampling error - systematic non-response eg by a particular socio demographic group. If people from group X are likely to refuse the census, they are just as likely to refuse the sample. Even with poststratification sampling to weight up the responses of those people from group X who you do persuade to answer your questions, you still have a problem because those might be the very segment of X that are pro-surveys. There is no real way around this problem other than to be as careful as possible with your design of instrument and delivery method.

In passing, this does draw attention to one possible issue that could make an attempted census less accurate than a sample. Samples routinely have poststratification weighting to population, which mitigates bias problems from issues such as that in my paragraph above. An attempted census that doesn't get 100% return is just a large sample, and should in principle be subject to the same processing; but because it is seen as a "census" (rather than an attempted census) this may be neglected. So that census might be less accurate than the appropriately weighted sample. But in this case the problem is the analytical processing technique (or omission of), not something intrinsic to it being an attempted census.

Efficient is another matter - as Michelle says, a well conducted sample will be more efficient than a census, and it may well have sufficient accuracy for practical purposes.

  • 1
    +1 This reflects a thoughtful and informative effort to understand the question and what motivates it. – whuber Jan 20 '12 at 20:55
  • I think the difference between a sample and an incomplete census is more than simply a higher propensity to weight the responses in a sample. After all, the weighting figures have to come from somewhere - a census, or higher-quality sampling. – Jonathan Jan 24 '12 at 19:38
  • I would really emphasize the ability to minimize nonresponse bias in a sample. Very few censuses have the ability to effectively go after nonresponse bias - even the US census struggles. Perhaps the only ones that can do it well are employee satisfaction surveys. It is much more cost effective to go after nonresponse in a sampled survey. – Jonathan Jan 24 '12 at 19:58
  • In a census, it might be (will be) to costly to quality control each interview/...! So often, the quality of data will be better in a sample than in a census. – kjetil b halvorsen May 7 '15 at 17:35

I think there are practical situations where a sample can be more accurate. For example, we did a study in a city in a developing country with a lot of people living in unregistered places and people constantly coming and going and being shy about responding. Trying to actually do a census would have required a Herculean effort, and given our resources it would have to have been done over the course of a couple months, when people would be coming and going. With a sample, we could spend more time making sure we got as close to full response as possible -- because we could explain what we were doing -- and we could do it over a much shorter time frame which would get rid of the problem of people entering and leaving the city.

SO I think the answer depends more on the logistics of what you are doing, and the various sources of non-sampling error.

In fact, another source was that our survey was complex and we had to train the interviewers, and finding and funding enough trainable interviewers in that country would be very difficult.

When sampling humans for surveys, samples often suffer from both sampling error (we're only getting estimates) and nonsampling error (e.g. people refusing to answer one's survey, not sampling to the sample frame one needs due to practical considerations such as cost, or inability to identify the population accurately in order to draw the sample). Done correctly, with high response rates, a sample is more efficient than a census. But it is incorrect to assume that no samples contain nonsampling error.

  • +1. Thank you for your reply, Michelle, and welcome to our community! – whuber Jan 20 '12 at 20:55
  • 1
    Hi whuber, it's nice to be here. Thanks for the welcome. :) – Michelle Jan 20 '12 at 23:33
  • 1
    @Michelle Just a little correction. Sampling error is error that results from not selecting the entire population--that is, error due to using a sample to infer population characteristics. Non-sampling error is everything else, including non-response, failure to create an adequate sampling frame, measurement errors, etc. – Brett Jan 21 '12 at 16:51

I think they key is in Peter Ellis' answer: "attempted". When you do sampling properly, you sweat the details of non-response, figure out strata and seek them out, etc. When you decide to do a census, it's easy to ignore those issues, since you're getting "everyone". Problem is, you're probably not getting everyone, but you're not thinking about who you're not actually getting.

There are also statistical issues with extremely large samples (as a proportion of the sampled population). I'm not sophisticated enough to understand them, but at a minimum you have problems with variance calculations. (Packages like R's survey compensate for such things in large subpopulations of a survey, and that's where I first learned about this.)

As a secondary issue, if non-sample error includes issues due to quality control at various steps in the process, having enormously more data (census) would make it much harder to have the level of quality control that you would have (with the same resources) on a smaller set of data (sample).

Imagine if you had the resources (financial and personnel) that the US Census Bureau used for a census, but you were only doing a survey of 1,000 random adults. I think you'd have much better quality control and much better analysis of the issues involved and of the data itself.

I thought the reason sampling can be (not is) more accurate than census actually did have one component that is attributable to the nature of a census versus a sample, and which can be attributed as the cause for a census potentially having greater bias (obviously non-sampling, by definition): in a census, the population number is generally unknown. So minimizing or controlling for non-response bias is a good deal more difficult than doing so with a sample of known size.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.