5
$\begingroup$

I have two datasets, both from the same population: The samples from the first survey are quite representative of the underlying truth. However, the second survey comes with a change in distribution due to sample selection bias.

If I merge the data and assign a class ('surveyA', 'surveyB') to each instance, it should be possible to predict from which survey an instance comes from (because of a biased distribution in 'surveyB'). Is it good practice to simply build a model to predict and remove instances that make this classification possible?

What are ways to "correct/remove the bias in" the second dataset? How can I achieve 0.5 accuracy in classification (assuming both datasets are equally large)?

edit: Both datasets represent surveys on political participation. SurveyB contains the data of probably more politically interested people, since they've participated in the first place. SurveyA can be assumed to be representative of "all people", don't ask me why.

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ Could you say little bit more about the bias. I.e. in addition to, "sample selection bias". These terms are quite broad. $\endgroup$
    – Jim
    Commented Jun 24, 2018 at 11:25
  • $\begingroup$ bias correction in surveys is a big field and people do a lot of different things, and without more details it's still hard for me to tell what exactly you're looking for here. Do you want to correct a statistic of one variable computed from survey $B$ by appealing to data in representative survey $A$? Or do you want to use auxiliary data like from the US national census? E.g. are you in a situation where you get a biased estimate of something like $P(\text{support candidate C})$ from survey $B$, and you want to weight that via the data in survey $A$ or with the census so that it's unbiased? $\endgroup$
    – jld
    Commented Jul 2, 2018 at 16:08
  • $\begingroup$ Exactly. I get a biased estimate of P(candidate turns out to vote) from SurveyB, and I need to "correct" or weight that using SurveyA. $\endgroup$
    – Laksan Nathan
    Commented Jul 2, 2018 at 18:49
  • 1
    $\begingroup$ I'd look into calibration. One well cited paper is this one by Deville and Sarndal from 1992. That particular one calibrates according to known population totals but I've seen other places where estimated totals are instead used, which is exactly your case (i.e. you'd calibrate the estimate from survey B according to the estimates from the more reliable survey A) $\endgroup$
    – jld
    Commented Jul 3, 2018 at 4:26

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.