2
$\begingroup$

I'm working with a large, national survey that was collected using complex survey methods. In order to make my results representative I need to account for sample weights and other survey design features (e.g., sampling strata). I'm new to this methodology, so apologies if the answers here are obvious.

Some of my models involve only a subset of the data (e.g., only female participants).

I have one questions:

Do I need to adjust the sample weights to reflect the fact that I am only analyzing a subsample (e.g., females)? My understanding is that not adjusting the weights can bias results (the standard errors in particular).

$\endgroup$
1

3 Answers 3

2
$\begingroup$

Yes you do need to use the weights. You do not adjust the weights, rather by using the weights, you adjust for the complex design of the survey to obtain efficient and unbiased estimates of the parameters of interest. If you ignore the weights, the analysis will most often be biased, or it may be inefficient. Getting the wrong standard errors doesn't mean the estimate is biased, but it can be conservative (or anti-conservative). For instance, if I wanted to know the incidence of a rare disease, I may conduct an SRS, then collect an additional sample from a high risk subpopulation. If I only analyze my SRS, it will be unbiased but very inefficient. By inverse probability weighting by the probability of randomly sampling a high risk individual, I can get a much tighter confidence interval for the prevalence of disease.

In R, the survey package has methods for calculating mean differences and GLM estimates from complex designs.

$\endgroup$
2
$\begingroup$

The correct analysis is equivalent to setting weights to zero for observations outside the subpopulation, then analysing the whole sample. So in that sense, yes, you adjust the weights.

You shouldn't actually do it that way; you should use the survey package, where the [ and subset functions are set up to do the right thing with subpopulation/domain analysis. Or if you are a tidyvert, use the srvyr package as an interface.

$\endgroup$
1
$\begingroup$

As said by the other answer, you must use the weights to account for the unequal sampling probabilities in this design.

There might be cases in which each observation in your data are self-weighted, in this case one is often tempted to ignore the sampling design as your point estimates are exactly the same before and after accounting for it. However, standard errors change due to complex sample designs and you should take this into account to build confidence intervals and hypothesis testing.

However, it will depend on your analysis whether or not to actually use the sampling weights. For regression analysis, it might not be necessary, as explained here. Maybe tells us more about what you're doing to give you some more guidance?

The survey package on R is the best way to manage this. You must declare your sample design and then use svy commands to compute statistics and run models. There is one caveat regarding the use of survey and the calculation of average partial effects of a general linear model with the marginspackage, however. If you feel like you need to do this kinds of things, maybe tells us more to see what we can do to help.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.