I am hoping to use some self selection survey data (it's from those awful things that pop up on the start page of a website asking if you have 5 minutes to spare).

This is for business decision making so obviously I don't want to overstate the importance of any findings.

I have found one public access paper that deals with "Heckman Corrections" here:- http://onlinelibrary.wiley.com/doi/10.1111/1467-6419.00104/pdf

Am I right in thinking that there are not any statistical methods in common use for correcting for self selection bias?

Are there any simple ones that would be useful for a quick and dirty decision making process, even if they're not perfect?

  • $\begingroup$ I appreciate that this is something of a "soft-question" but there is no tag on this part of stackexchange to cover this, else I would have used it. $\endgroup$ – Simon Hayward Oct 31 '12 at 14:22

It seems to me that the best you can do is to first lay out the demographic that you did capture, and make sure that the readers of your report are aware that the conclusions only apply that that particular demographic, and you simply don't know much about whatever your target population is.

If you have enough external data about your target population, you might present that as well, to give an idea of how representative your sample is of the population. For example, 50% of your population have graduate degrees, but only 20% of the respondents did. But my guess would be that you don't have external data about your population.

If you actually do have enough external data, you might be able to weight your results based on how representative your sample is of your target population. So if you know that your target is 66% male, but 85% of respondents were male, you could down-weight male responses. Though that's still making a bunch of assumptions, especially if your sample is small.

You could try to model response versus non-response, but again that would require external data about your population. I believe that hierarchical (multi-level, mixed-effects) models might be helpful.

I'm not a survey guy, so may be way off, but wanted to see that you have at least one answer.

| cite | improve this answer | |
  • $\begingroup$ Ha, thanks! It's much appreciated. I won't mark as accepted just yet, let's see what other people say... $\endgroup$ – Simon Hayward Oct 31 '12 at 15:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.