Do survey sample standard errors take into account/correct for anything else besides finite population?

Question

Regular standard errors are biased when the data comes from a survey sampling design.

The article "Wait Wait, Don't Tell Me... You're Using the Wrong Proc!" explains that the bias is due to the violation of the independence assumption because the target population from which survey sample data is drawn is finite.

Example 4 of "Guidance for use of weights: an analysis of different types of weights and their implications when using SAS PROCs" simulates a sample of data that is 1% the size of the target population. The difference between the regular standard errors (WOLS/ML) and the survey standard errors (EE) is quite large.

Table 3

Monte Carlo mean, empirical, WOLS/ML and EE SE
WOLS/ML                    EE
Mean 9.14                  9.14
SE (WOLS/ML): 1.36
SE (empirical): 0.726      SE (EE): 0.726

EE, estimating equation; ML, maximum likelihood; WOLS, weighted ordinary least squares.

With a target population so much larger than the sample, it seems like the infinite population assumption would be OK or that the two methods for computing standard errors would not be very different.

Do survey sample standard errors take into account/correct for anything else besides finite population?

Answer 1

Yes, indeed they do.

There are typically four things involved

• negative correlations due to finite population
• negative correlations due to stratified sampling
• positive correlations due to cluster sampling
• unequal (and large) weights

The first one is unimportant unless the sampling fraction is large. The other three are important even for large sampling fractions, especially the last two.

Cluster sampling makes the true variance larger, potentially much larger, so it is important. Unequal weights mess up the computation of standard errors -- to a moderate extent for linear regression, but to a major extent for some categorical data analyses if your software thinks they are frequency weights (see here for types of weights)