A survey to a representative sample of the population has been submitted. This sample is composed by 4.400 individuals. However, only 2003 individuals have completed to survey. I want to verify if this subsample (2003 individuals) is representative of the population with regards to some key variables. Concerning this issue, I have two questions:

  1. What is the most suitable method? I was thinking to t-test for continuous variable and chi-square goodness of fit test for categorical variable (or is more appropriate z-test for the latter?)

  2. Should I compare who has completed the survey with who hasn't (2003 vs. 2397), or alternatively who has completed the survey with the whole sample (2003 vs. 4400)?

Thank you

  • $\begingroup$ What information do you have about the $2397$ people who did not complete the survey? $\endgroup$ – whuber Jul 29 at 13:47
  • 1
    $\begingroup$ You're right, I haven't specified. Some informations are available for all the 4400 individuals (age, income, job, gender etc.), as they are provided by the national statistics institute. But for 2397 people we don't have the answers related to a specific survey. The representativeness test is based on the information available for all the 4400 people $\endgroup$ – Andrea Jul 29 at 20:22

One approach would be to take the information you do have for all 4400 individuals and treat the situation as a logistic regression: model the probability of completing the survey as a function of the available information. That analyzes all the data together, avoiding the problem of multiple tests and handling different predictors in different ways. As you have a large sample with over 2000 cases in your minority class (the 2003 that completed the survey), such a model could be quite complex without much risk of overfitting, including: interactions among the predictors, splines instead of simple linear functions for continuous predictors, etc., up to something like 100 predictors and combinations.

If the model as a whole isn't significant then you can say that there was no difference in response rate based on the information you have. If some of the predictors or combinations are related to response rate, that gives a hint as to how to proceed with analysis and interpretation.

That approach should satisfy the concerns embedded in your two specific questions in a simple, efficient way.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.