Does anyone know a method for comparing two variables with a chi square test if the variables are from different surveys with different svydesign() statements? I am looking to test a difference in a variable distribution across two waves of a survey, but the svychisq() statement is limited to one design object.

Is it legitimate to stack the two variables in a new data.frame, create a new svydesign statement with the collective weights and then run the test?

  • 2
    $\begingroup$ This should be migrated to CrossValidated stats.SE website. I'll wait for it to be migrated, but start reading this in the mean time: citeulike.org/user/ctacmo/article/8898414 $\endgroup$
    – StasK
    Jul 17 '13 at 13:13
  • $\begingroup$ @StasK any link without a paywall? $\endgroup$ Jul 17 '13 at 14:05
  • $\begingroup$ @AnthonyDamico, ask Statistical Society of Canada :-\. It may be on Wu's page, too. $\endgroup$
    – StasK
    Jul 17 '13 at 23:18
  • 1
    $\begingroup$ What do you mean by "compare"? Are these continuous variables, ordinal variables, nominal variables? There is not enough in your question to be answered properly. $\endgroup$
    – StasK
    Jul 24 '13 at 16:20
  • $\begingroup$ @StasK, thanks for help, just to be clear, this comparison is for ordinal as well as continuous variables $\endgroup$
    – david rae
    Jul 24 '13 at 16:49

If you are going along the path of stacking the data sets together, then you should define super-strata corresponding to the two data sets/waves, so that svydesign() knows that they are independent. Thus your new svydesign will have strata = cross of year and strata, the PSUs from the original designs, and the weights from the original designs.

As I suggested in the comment, other ways of combining estimates and tests have been proposed in the literature. Wu (2004) uses empirical likelihood based on common variables between the two data sets.

For continuous variables, ideally, you would want to use Kolmogorov-Smirnov test with "flat" data, but I don't know whether extensions for it work for survey data; I doubt it. So you may have to convert your continuous variables to ordinal ones into say $[\log_2(n)]$ percentile groups or equal width bins of the variable range (where the above function of the sample size is a commonly used number of bins for a histogram), and apply the Rao-Scott $\chi^2$ to them.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.