I'm working on a revision of a manuscript. I have a longitudinal cohort of adolescents that were followed throughout junior high school. One of the reviewers considers the attrition to be considerable, and asks me to reweight the sample to reflect the census on certain characteristics, so that the results can be compared to the general population that the sample aims to reflect.

I understand the idea, but how do I go about to do this? I mean, say that I have a divorce rate (one of the characteristics suggested by the reviewers) of 10 percent in my sample and 15 percent in the relevant general population, how do I reweight my sample to reflect this? Is is it that simple that the 10 percent with divorced parents get a weight of 15/10 = 1.5 and the 90 percent with parents that are not divorced get 85/90 = 0.94, and then I just use this weight in the calculations? Do anyone have materials to good tutorials regarding this issue?


This sounds like a reasonable suggestion, but if you are not very familiar with the way survey weights work, this may be rather complicated and tedious. The methods to align samples with known populations have long existed in survey statistics, going back to at least mid 1970s when the methodology of post-stratification was first formalized, and 1990s when the more general methodology of calibration was. Either methods means that that standard errors have to be calculated in a special way, so you have to be careful about these.

Post-stratification proceeds by breaking the population and the sample into non-overlapping classes, and assigning weights to the sample units in such a way that the weights sum up to the known population totals in each class. This begins to break down when you have more than two or three dimensions to adjust upon (say, age, gender, and race/ethnicity -- you need to know the totals like "number of black females age 18-34"), as then you can run out of sample members to populate your classes. Typically, it is desirable to have 50+ sample units in each class for post-stratification to work reliably. Computationally, post-stratification is a single step:

$$ w_i^{\rm PS} \leftarrow w_i^{\rm source} \frac{\mbox{Population total in cell $c : i \in c$}}{\sum_{j \in c} w_j^{\rm source}} $$

Calibration is a more general approach that allows to adjust the weights to make the sample agree with the population one several dimensions, but not necessarily on all cross-tabs. (With the above example of age, gender and race/ethnicity, you need separate totals by age groups, by gender, and by race, but you don't need the three-way tables. For more on the fine points of the differences between the methods, see my paper in Survey Practice: Kolenikov (2016)). Procedures for weight calibration are usually iterative, with the simplest one being, effectively, post-stratification on each variable at a time in a cycle, called raking. I implemented it in Stata; implementations are available in SAS, R and SPSS (although the latter did not match the other three in my cross-testing across the possible platforms).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.