Complex survey data is that typically found produced by the National Center for Health Statistics (NCHS) or the NSLY; it typically contains information on PSU, strata, and weights. To make nationally representative samples, one would traditionally perform a weighted regression that accounts for the sampling design by Taylor linearization (i.e. the survey analog to Huber-White errors).

I'm interested in matched analyses (e.g., King's MatchIt program) as a manner in which to improve causal inference. What remains unclear from a first look is: (1) what criteria should be used to determine when matched analyses are appropriate with complex survey data; and (2) how such matched analyses ought to account for weights and/or survey sampling.

My understanding of (1) is that there is nothing different about these analyses than any other, but that it might/must improve inference and efficiency when the number of matched cases is small. As regards (2), my understanding is that common recommendations suggest including weights, and not the sampling design, in the matching (e.g., a weighted logistic regression to develop propensity scores) and not the later causal inference.

Should the sampling structure (e.g., PSU, strata) not be taken into account? Any references, suggestions, confirmations, or contradictions of what is above would be welcomed.

  • 1
    $\begingroup$ Hi there, it is not usual to use survey data to make casual inferences, because of the lack of control of variables. One normally uses an experiment for investigating casusality. You can, however, use survey data to investigate correlations. $\endgroup$
    – Michelle
    Mar 7 '12 at 17:03
  • $\begingroup$ Excellent question. I often wonder about how this is supposed to work and it invariably hurts my head. $\endgroup$ Mar 7 '12 at 17:24
  • $\begingroup$ @Michelle Certain surveys (e.g., NSLY) are longitudinal in design and may be better able to approximate causality. I'm not suggesting these techniques allow one to ascertain true causal inference. More advanced analytical techniques (e.g., matching, g-estimation) can be used to approximate an ideal randomized control trial and potentially reduce bias by functional form of the models specified. Even RCT analysis requires untestable assumptions (see the work of Hernán, Robins, etc. at Harvard) and common techniques (e.g., intention-to-treat) may be inappropriate $\endgroup$
    – user6521
    Jul 6 '12 at 3:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.