I have two datasets, both from the same population: The samples from the first survey are quite representative of the underlying truth. However, the second survey comes with a change in distribution due to sample selection bias.

If I merge the data and assign a class ('surveyA', 'surveyB') to each instance, it should be possible to predict from which survey an instance comes from (because of a biased distribution in 'surveyB'). Is it good practice to simply build a model to predict and remove instances that make this classification possible?

What are ways to "correct/remove the bias in" the second dataset? How can I achieve 0.5 accuracy in classification (assuming both datasets are equally large)?

edit: Both datasets represent surveys on political participation. SurveyB contains the data of probably more politically interested people, since they've participated in the first place. SurveyA can be assumed to be representative of "all people", don't ask me why.

  • 2
    Could you say little bit more about the bias. I.e. in addition to, "sample selection bias". These terms are quite broad. – Jim Jun 24 at 11:25
  • bias correction in surveys is a big field and people do a lot of different things, and without more details it's still hard for me to tell what exactly you're looking for here. Do you want to correct a statistic of one variable computed from survey $B$ by appealing to data in representative survey $A$? Or do you want to use auxiliary data like from the US national census? E.g. are you in a situation where you get a biased estimate of something like $P(\text{support candidate C})$ from survey $B$, and you want to weight that via the data in survey $A$ or with the census so that it's unbiased? – jld Jul 2 at 16:08
  • Exactly. I get a biased estimate of P(candidate turns out to vote) from SurveyB, and I need to "correct" or weight that using SurveyA. – lnathan Jul 2 at 18:49
  • 1
    I'd look into calibration. One well cited paper is this one by Deville and Sarndal from 1992. That particular one calibrates according to known population totals but I've seen other places where estimated totals are instead used, which is exactly your case (i.e. you'd calibrate the estimate from survey B according to the estimates from the more reliable survey A) – jld Jul 3 at 4:26
  • I will have a look at it. Thx ! – lnathan Jul 3 at 8:29

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.