Say you want to give a survey to people from a population and assume that

  1. people who volunteer to take the survey are not necessarily representative of the population ("volunteer bias") and

  2. you don't actually know the underlying population demographics.

To be more specific, say you are trying to measure the gender balance in a certain population, and for example, the true balance is 50/50 (which you don't know), but the sample you get among survey takers is 55/45 (more females). Is there anything that can be done?

  • $\begingroup$ Even for random samples the sample proportion of females will usually not be exactly 0.5 and it could be 0.55 for a small sample. If you don't know the population proportion how would you know what to adjust? $\endgroup$ – Michael Chernick Mar 2 '18 at 21:23
  • $\begingroup$ @MichaelChernick Well, that definitely is a problem, but for the sake of argument, let's say you have large enough samples that you could get arbitrarily close, if there was no other bias involved. Personally, I don't think there's anything that can be done, it's just not clear to me how to deal with this kind of bias. $\endgroup$ – thecity2 Mar 2 '18 at 21:25
  • $\begingroup$ I found a similar question here: stats.stackexchange.com/questions/204502/… $\endgroup$ – thecity2 Mar 2 '18 at 21:29
  • $\begingroup$ You can test the null hypothesis that p=0.5. If you reject you have a high level of confidence that the proportion of females is different from those for men but you can't know exactly what it is. $\endgroup$ – Michael Chernick Mar 2 '18 at 21:52
  • $\begingroup$ @MichaelChernick Gender was just one example. It becomes harder when you have to deal with, say, race/income/education level, etc. I wouldn't know any of these except to assume that they are similar to US population. $\endgroup$ – thecity2 Mar 2 '18 at 21:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.