When is subsetting survey data prior to analysis NOT a problem? Survey researchers are typically advised to not subset their data prior to analysis because it will produce incorrect variance estimates. My understanding of the inferential issue is that removing observations rather than setting their sampling probability to 0 is only problematic because it will lead to incorrect numbers of PSUs.
However, I'm not sure then how to reconcile this with other advice I've found, which is that it is okay to pre-subset when there is no clustering or stratification in the design. On the one hand, subsetting could theoretically remove all observations from a PSU and that information would be lost. On the other hand, in non-clustered/stratified sampling, each observation is its own PSU and therefore dropping observations would be problematic on the grounds that it removes PSUs from the sampling information.
Can someone provide an explanation for when it IS okay to pre-subset rather than set weights to 0? (or INF depending upon how the weights are provided)? Would appreciate some demonstration via reference to the variance estimation of the H-T estimator, but okay without.
 A: There are two issues here:

*

*When is it desirable to remove some records from the data set before analysis?

*When doesn't it make any difference?

When to remove records before analysis
You want to remove records before analysis when the records are not part of the sample.  Consider, if you will, the NHANES series of surveys. These recruit a whole lot of people. Some of the people only get asked questions; some also get a full medical exam; some also get blood drawn for biochemical tests.  If you want to analyse the blood tests, you need to use the blood tests sample. People who were not chosen for the medical exam/blood tests are not part of the sample and should not be included.  Operationally, they will have missing values for the blood-test-sample weight variable.
You can think of this as just a data-compression hack. There are three (or maybe more) overlapping but conceptually distinct samples being taken. They are stored in the same data file for convenience, so some of the records in that data file may not belong to the sample you are analysing. This has nothing to do with the Horvitz-Thompson variance estimator; it's just bookkeeping.
When subsetting (potentially) doesn't matter
Once you have identified the sample, your analysis should ideally use everyone in the sample, even when you are interested in a specific subpopulation. If you want, say, the mean for people over 60, you estimate by setting the sampling weight to zero for people who are not over 60.
You will get the same point estimates by just subsetting to the subpopulation of interest. In general you will not get the same standard error estimates. But sometimes you will. There are two situations where you get the correct standard errors by subsetting.
The good one: strata. Strata behave like (are?) separate survey samples conducted for specific subpopulations.  If your subpopulation of interest is a stratum or a set of strata, it is valid to subset to just those strata before analysis.  In terms of the variance estimator, the variance of an estimated population total is just the sum over strata of the variance of the estimated stratum totals, because the estimated stratum totals are independent.
The slightly dodgy one: all the PSUs. This is the one that depends on the details of the variance estimator. Suppose your survey is (or is approximated by) a single stage of stratification and clustering, with clusters sampled 'with replacement'. This is a common public-use data format, eg for NHANES. The Horvitz-Thompson variance estimator for a total is
$$\sum_{h\in\textrm{strata}}\sum_{i\in\textrm{PSUs}} (Z_{hi}-\bar Z_h)^2$$
where $Z_{hi}$ is the weighted sum in PSU i, stratum h.
If every PSU appears in the subpopulation, then $Z_{hi}$ is a non-empty sum for every PSU, so $Z_{hi}$ is the same whether you (incorrectly) drop records outside the subpopulation or (correctly) use a weight of zero for records outside the subpopulation.  If PSU $i$ in stratum $h$ does not appear in the subpopulation and you (incorrectly) drop records, you don't have a $Z_{hi}$. If you (correctly) use zero weights, you get $Z_{hi}=0$.
If you have a more complicated design -- eg the actual four-phase NHANES design -- you would need to worry about smaller changes when some secondary sampling units or tertiary sampling units or whatever weren't in the subpopulation. But if you're working with the actual four-phase NHANES design you probably know the answer already.
