1
$\begingroup$

Please excuse me if this has been asked before, I searched and couldn't find what I was looking for. The basic setup of my question is this - I have data on all of the students for a school. I then conduct two different surveys (for example, a paper-based survey and an internet-based survey), that try to target all of the students. This results in 2 groups of respondents. So far, I have done chi-squared test of goodness of fit on various demographic features (e.g. age bracket distribution) as well as measured effect size, comparing each survey respondent group with the overall student population. For example:

chi squared and p-value for survey 1 and population: 1057.11668626 3.95209518794e-225

Cramer's v: 0.219299550203

chi squared and p-value for survey 2 and population: 20750.6706798 0.0

Cramer's v: 0.971610827539

I want to say that one group of respondents is more representative of the general student population than another. Based on the above results, for example, I'd say that the distribution in age for the respondents of survey 1 is more similar to the overall student population, than "" of survey 2.

Is there a more proper/better way to compare how representative various sample groups are with a population?

Improve this question
$\endgroup$
4
  • $\begingroup$ Maybe evaluate the KL-Divergence between the empirical and theoretical distributions? $\endgroup$
    – LE Rogerson
    Commented Nov 7, 2016 at 15:58
  • $\begingroup$ Is it possible that there are students who responded to both surveys? Presumably yes. If so, do you have a way to tell which paper respondents represent the same person as which Internet respondents? $\endgroup$
    – Kodiologist
    Commented Nov 7, 2016 at 16:48
  • $\begingroup$ @Kodiologist, Hi, yes it is possible to tell which students responded to both surveys (based on student IDs). $\endgroup$
    – firefly454
    Commented Nov 8, 2016 at 19:33
  • $\begingroup$ @LERogerson, Thanks for the suggestion, I've never heard of KL-Divergence. Is this a normal use of the metric (as in, if I tried to publish these results, would people accept this analysis)? $\endgroup$
    – firefly454
    Commented Nov 8, 2016 at 19:44

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.