2
$\begingroup$

Working off a fairly limited statistical understanding, so apologies in advance if this question is ludicrously basic. This has to have happened to other people who have successfully solved this issue, but searching for what I think are relevant terms is getting me things like "Karlin's Conjecture for Random Replacement Sampling Plans," and I'm starting to panic.

Attempting to contact 1000 people for a survey. Plan was to get a list of the target population, draw a random sample of 1000, then contact those people. Problem was that the original list contained people who weren't actually part of the population, so our random sample of 1000 only contained around 650 people we could actually contact. For time & budgetary reasons, filtering the entire original list so it only contains our target population is impossible.

How do we fix this? One option is to take a second random sample of 1000, filter for the people who actually are in our target population, then contact everyone who fits the bill (guessing around 1300 total from both samples?).

Another option is to take a second random sample of 1000, filter for the people who actually are in our target population, take the resulting list of 1300 and randomly contact 1000 of them.

A third option is to throw the original sample back in to the list, take a larger initial random sample (1500-2000 people?), and contact everyone from that sample who meets our criteria. For time and sanity reasons we'd like to avoid this if possible.

All of these feel somehow "off" to me, given my incredibly basic statistical education. So, gurus: what am I missing? Is there a fourth option that makes more sense? Will our results be statistically valid given any of the above approaches? And does this affect our analysis in any way?

$\endgroup$
6
$\begingroup$

This is a very common situation, and usually one just plans for a larger initial sample size, so that once ineligible subjects are thrown out, the desired sample size is reached. In your case it appears that ~65% of the original list qualifies. So to get 350 more qualifying subjects you need to draw a random sample of about 540. In fact, depending on how you do the filtering, you could keep drawing additional random samples of size 1 until you get the required number of qualifying subjects.

$\endgroup$
  • 1
    $\begingroup$ +1 It's a straightforward sampling problem and this is a straightforward solution. $\endgroup$ – Cyrus S Dec 16 '10 at 2:36
1
$\begingroup$

I hate answering a question with a list of questions, but I hope thinking about some of these issues will give you some insight as to the best way to proceed. The first two questions are mainly for clarification as it wasn't clear from reading your question:

  1. Have you already obtained a list of 1000 people?
  2. Have you already collected 650 samples?
  3. If you have already collected your survey data, did all 650 people who qualify for your survey actually complete it? Even if they qualify, I have to imagine some # of those people will not complete your survey for a variety of reasons. You'll need to account for some additional attrition to reach your goal of n = 1000.
  4. What are the costs associated with obtaining a larger list?
  5. Are the costs of collecting the data sufficiently high enough to dissuade you from obtaining "extra" sample above you target of n = 1000?
  6. Do you really need n = 1000? What are the negative effects of n = 900? n = 700?
  7. Are there high start up costs to collecting the data in phases?
  8. How is your survey administered? Can you build in the filtering system into the survey? i.e. "If answer to question 1 == yes, then terminate"?
  9. Does your survey contain any temporal elements that may introduce bias if you conduct the survey in two/three/four phases? i.e. are people going to respond or behave differently if they answer the survey in December vis a vis March?

Without knowing more of the specifics of your survey and the population in question, the most straight forward approach in my opinion would be to obtain a random sample that is sufficiently large enough to account for the people who won't qualify and those who won't take your survey. This sounds like option three that you described above. If you already have n = 650, then I would recommend obtaining a list large enough to collect the remaining 350 samples. Does your survey contain demographic data or other benchmarks that you can use to compare across groups (waves of data collection) to see if you have introduced any bias into your data?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.