# Tag Info

11

The term calibration'', as applied to survey weights, appears to have been coined by Deville and Sarndal (1992). They put an umbrella on a bunch of different procedures that used the known population totals: $$\sum_{i \in \mathcal{U}} Y_i = T_i$$ where $Y_i$ is a vector of characteristics known for every unit in the population $\mathcal{U}$. For ...

5

This sounds like a reasonable suggestion, but if you are not very familiar with the way survey weights work, this may be rather complicated and tedious. The methods to align samples with known populations have long existed in survey statistics, going back to at least mid 1970s when the methodology of post-stratification was first formalized, and 1990s when ...

5

If people say they have post-stratified weights, it does not necessarily mean they implemented post-stratification, proper (as in, rescaled the weights in each demographic cell to the known population total). About 80% of usage that I hear of "post-stratified weights" actually refers to calibrated weights (i.e., rather than trying to adjust each and every ...

5

This is a good detect. You are referring to positivity assumption. It requires that there be both exposed and unexposed participants at every combination of the values of the observed confounder(s) in the population under study. Positivity violations occur when certain subgroups in a sample rarely or never receive some treatments of interest. There are many ...

5

If confidence and p-values are calculated using different methods, then this situation is possible. The $z$-test you have there is a Wald's $z$-test: $$z=\frac{AME}{SE}\quad\text{then}\quad p=\min{\Big\{1,2\times\big(1-\Phi(\lvert z\rvert)\big)\Big\}}$$ # in R syntax: min(1, 2 * (1 - pnorm(abs(AME / SE)))) For the confidence intervals to correspond to the ...

5

SPSS LOGISTIC does not handle sampling weights correctly for computing standard errors. If you have weights $w_i$ for each observation, SPSS will work out the loglikelihood contribution $\ell_i(\beta)$ for each observation, and maximise the weighted sum $\hat\ell(\beta) = \sum_i w_i\ell_i(\beta)$. So will R. The point estimates will agree exactly. SPSS, ...

4

It is an interesting twist to find a case where the demographic data will be seen as less reliable as the behavioral data. There isn't much good advice on how to select the calibration variables, other than they should correlate with both the (non-)response process and the variables of interest. The reasoning behind the widespread use of demographic ...

4

svr is a user-written alternative to Stata's native svy, which uses Taylor series linearization. The later can now accommodate multilevel mixed-effects complementary log-log regression, GLMs, vanilla and ordered logistic/probit, Poisson and negative binomial regression, and parametric survival analysis models. You can read about replication weights in the ...

4

(Update: There isn't very much work yet on "modern" ML methods with complex survey data, but the most recent issue of Statistical Science has a couple of review articles. See especially Breidt and Opsomer (2017), "Model-Assisted Survey Estimation with Modern Prediction Techniques". Also, based on the Toth and Eltinge paper you mentioned, there is now an R ...

4

Let's start with this unstrat_design = svydesign( id = ~ 1, strata=NULL, FPC = ~FPC, data = srswor %>% mutate(FPC = 1-sample_size/P) ) ## Warning in svydesign.default(id = ~1, strata = NULL, FPC = ~FPC, data ## = srswor %>% : No weights or probabilities supplied, assuming equal ## probability unstrat_design ## Independent ...

4

lifelines' author here. I am not familiar with the inner workings of svycoxph, so I can't comment authoritatively on the similarities of it and CoxPHFitter. Looking at its docs page, there are some difference that lifelines doesn't handle, for example accounts for the reduction in variance from stratified sampling and the increase in variance from having ...

3

Weights are used when you want to generalize to the underlying population. The analysis of response/nonresponse process is basically an analysis of how well your particular protocol worked. As such, you are analyzing your existing full sample of respondents and nonrespondents, and you are not trying to generalize. So you don't need weights, and can/should ...

3

In version 3.31-3 or later, I have added an option method="xlogit" to svyciprop() that appears to reproduce the SUDAAN results. As far as I can tell, the SUDAAN formula is not documented, but published examples show it is the same as SPSS uses, and that is documented. The algorithm is Estimate the mean and standard error as usual Apply a logit transform ...

3

You may be confusing the population vs. the sample moments, on one hand, and the population variance and the variance of the estimate of the mean (which involves the strata variances... unfortunately... as well as a lot of other stuff like finite population corrections $1-f_h$). The population moment of order $k$ is, obviously (with some abuse of notation ...

3

It looks like you've done all you could. The strata with the high non-response rates apparently do not constitute a large portion of the population. In retrospect, I would have suggested a smaller sample, with pilot tests, and more time devoted to follow-up of a random sample of non-responders. Here are my thoughts on what you should do: Reweighted ...

3

say you run a survey and get 1,000 responses. maybe you conducted your survey via cell phone and older people don't have cell phones at the same rate as younger people. so 5% of your survey respondents (N=50) were senior citizens but maybe according to the united states census bureau, 15% of americans are actually senior citizens. and let's say respondent ...

3

You can compare variances from first principles, i.e., by calculating the variance as the difference between value-squared and mean-squared. webuse nhanes2, clear gen bpsystol_sq = bpsystol* bpsystol svy : mean bpsystol*, over( female ) * estimated variance in group 0 nlcom ( _b[bpsystol_sq:0] - _b[bpsystol:0]*_b[bpsystol:0]) * estimated variance in group 1 ...

3

bad news :) your standard errors, confidence intervals, and tests of significance will be incorrect if you do not account for the relationship between the original and post-stratified weights. i believe you can back-calculate the original sampling weights if you have the sampling clusters (although you'd have to invest a lot of time reversing the method in ...

3

The sampling weights are designed to account for the non-simple random sample nature of your sample. Therefore, they are just as needed in one form of regression as another. Exactly how to do this may be complicated; e.g. in SAS there is PROC SURVEYLOGISTIC to deal with various sorts of samples. In R there is the survey package which I think does similar ...

3

Here are some explicit ways that the model-based estimator can be biased Heteroskedasticity. Let X be binary and Y be continuous. We know that linear regression of Y on X reproduces Student's t-test (the equal-variance t-test), and we know that if the variance of Y is different between the X groups that the t-test has the wrong level. If the smaller group ...

3

There are two separate issues here. Sometimes, including with NHANES data, you do need to subset before defining the survey design object, because not all the records in the data set are part of the sample you are analysing. In NHANES, everyone in the data file will have a health questionnaire, but only a subset will have a clinical examination, and there ...

3

The point of survey weights is to make your sample closer in distribution to the target population. You need to know what the corresponding table for the whole population would look like. Suppose that in the population 12.32% of people are Male aged 18-29. In your sample there fewer people in this category, so you want to give them extra weight. Each person ...

3

Executive summary: this is much harder than you would expect, and there is neither a standard implementation, nor even an accepted estimator. Let's fix some terminology. In a survey, you have primary sampling units, secondary sampling units, and so on for potentially multiple stages. In a mixed model you have top-level clusters, possibly clusters at ...

3

As @Cam.Davidson.Pilon says, there are some differences. But not a lot. There's one big difference: svycoxph can account for stratified sampling, which coxph and (apparently) lifelines cannot. This can be important if you have good stratification, but ignoring stratification is conservative. Another difference is that svycoxph can distinguish between ...

2

The most commonly used non-response adjustment method is that of weighting classes. In this method, you split the population/the sample into non-overlapping groups that (you think) have similar response propensities. These groups can, and most of the time do, cut across strata. Then you adjust the weights within each of these groups so that the sum of the ...

2

There are other uses of the term "calibration." For instance in this CV thread, Frank Harrell discusses it in the context of determining model fit: The key thing to check first is the model's calibration, either using the bootstrap to correct for overfitting or using a huge independent sample not used for model development or fitting. Understanding ...

2

In R you can calculate weighted variance for each group using function "wt.var" from the package "SDMTools". As a result you'll have two weighted variances - var1 and var2 F statistic equals var1/var2 And the degrees of freedom for that F value are WEIGHTED n1 - 1 and WEIGHTED n2 - 1. Done

2

most nationally-representative survey weights are generated with those certain demographic characteristics (e.g. age, race, gender) as a part of their fundamental construction. unless you have a strong justification to stray from the weights provided to users of the microdata, you should err on the side of sticking with the survey weights. in r, this would ...

2

There is R code and output in the tests directory of the R survey package.

2

The weights only need to be used when trying to generate population estimates from the survey data. If you're interested in running behavioral models - as you seem to be - just use the raw data without the weights. As long as you've got a reasonable cross section of survey respondents in the sample that cover the behavioral responses of interest, and you're ...

Only top voted, non community-wiki answers of a minimum length are eligible