# Dealing with singleton strata in survey analysis

I am primarily using Stata for my survey analysis, but my question is also applicable to R or SUDAAN.

There are a number of different ways of accounting for singleton strata in survey analysis. Just to be clear, when I say "singleton strata", I mean that there is only a single sampling unit within one or more strata in my analysis. The problem this poses is primarily for variance estimation. Some sources recommend either leaving those strata out of the analysis entirely or merging that unit into another strata within the analysis. These methods can be justified in a number of scenarios, and have been discussed in detail in other online sources, but suffice to say that I do not find them appropriate for my data set. Another method is to generate replication weights (bootstrap or jackknife, etc.). This is possible for my data set, but would require extra work that I'm not convinced is necessary at this point.

Rather, I want to focus on the 3 methods you can feed into the singleunit("") command in STATA, or the options(survey.lonely.psu="") command in R. These methods are:

1) Certainty: this makes it such that a strata with a single unit does not contribute to the standard error estimates at that level of sampling. I believe this is equivalent to specifying a finite population correction of 1.

2) Scaled: this makes it such that the contribution to the standard error for that singleton strata is set equal to the average estimate for all of the other (multiple unit) strata. This is essentially mean imputation.

3) Centered: this calculates the standard error estimate for the strata based on the distance from the grand mean across strata rather than a strata mean.

In practice, when I try all 3 methods on my data, the differences are minimal. The largest difference in standard errors was 0.001, which is not something I am particularly worried about. However, I would still like to understand the theoretical differences between the methods in more detail before I decide which one to implement in the final analysis, even if practically there is no difference.

So, basically, what are the theoretical distinctions between these techniques? When might you choose "certainty" versus "scaled" versus "centered"? The one I see used most often when I search around the Internet is "certainty", but none of the sources have actually provided a justification for that. I would guess that "centered" is the most conservative of the 3 methods since it likely overestimates the standard error. Beyond that, though, I am at pains to think of any rigorous way of deciding between the methods.

Anyone have any suggestions? Or even know a citation where someone evaluates these methods in a simulation study?

Also looking for an answer to this. The closest I've found so far is from the R survey package manual:

Using "adjust" [centered in Stata] is conservative, and it would often be better to combine strata in some intelligent way. The properties of "average" [scaled] have not been investigated thoroughly, but it may be useful when the lonely PSUs are due to a few strata having PSUs missing completely at random.

The "remove" and "certainty" options [certainty] give the same result, but "certainty" is intended for situations where there is only one PSU in the population stratum, which is sampled with certainty (also called ‘self-representing’ PSUs or strata). With "certainty" no warning is generated for strata with only one PSU. Ordinarily, svydesign will detect certainty PSUs, making this option unnecessary.

Other sources (e.g. here) seem to suggest that selecting certainty is only appropriate when there is only a single "large" PSU in this stratum in the population, but also note that using approximate methods can give values that are too high.

This answer is from the point of view of R, but the issues are not primarily about the software, so it should apply generallly

"certainty" is not needed very often in practice. It's quite common to have certainty PSUs at stage 1 -- for example, the National Health Examination Survey has some large cities sampled with certainty. I don't have the list, but I'd expect New York, Chicago, and Los Angeles to be on it. When the PSU is sampled with certainty, there's no sampling uncertainty at stage 1, and the multistage variance estimator can just move on to stage two.

The reason you don't need this very often in practice is that you can just treat certainty PSUs as strata. That's easy and you don't have to worry about what's going on in the internal workings of your statistical software.

The other two are for situations where there is sampling uncertainty at stage 1 in that stratum. You now have a problem: no unbiased variance estimator exists. Basically, in that stratum, at stage 1, your variance estimator looks like a variance between the PSUs, and so it ends up as 0/0.

There are several things you can do here. One is just to zero out the contribution of that stratum to the variance; obviously not ideal, but easy, and if you have lots of strata it can't matter that much. That's what "remove" does. Numerically, it's the same as "certainty", but it's conceptually completely different, which is why it has a different name. "remove" is anti-conservative.

Another possibility is to pretend the strata are a random sample ('exchangeable', as our Bayesian comrades would say). In that case, you can replace the unknown variance contribution of this stratum with the average over all the other strata.

Yet another possibility is to discard the whole point of stratification and work out the variance by computing residuals from the whole-population mean rather than the stratum mean. It's using the stratum mean when you have just a single PSU that gives 0/0; the zero in the numerator is a sort of residual sum of squares involving PSU means minus stratum means, and when there's just one PSU those are the same. That's the "adjust" option. It's conservative.

Given these problems, why would you even have a single-PSU non-certainty stratum? It could be because of missing data or non-response. It could also be because the study designers wanted to use a single PSU per stratum to reduce standard errors even at the cost of not being able to get an unbiased estimate of them. That's what happens with some studies from the National Center for Health Statistics: they use one PSU per stratum, but in the public-use data they group the PSUs into pairs and pretend there were two PSUs per paired stratum.

If you know enough about the population, that's likely to be the best strategy. You look at the data and say, ok, this is Seattle, let's pretend it's in the same stratum as Portland. You get variances for that stratum from Seattle vs Portland differences. That's better than using Seattle vs whole-US differences (adjust) and probably better than saying the Seattle variance is the same as the average stratum across the country (average) and probably better than saying Seattle is just perfect and doesn't contribute any variance (remove).

The problem is that if you only have public-use data from some national survey, you won't have enough information to do that.