3
$\begingroup$

I have data coming from a sample that, for various reasons, don't follow the original sampling plan. Trying to calibrate this sample seems to be very difficult because deviations are hard in some cases.

At first the design aimed to include a number of small regions but not all of them have participated in the study. If we consider bigger areas we have data from all of them but the distribution in the sample is very different from the population (even for post-calibration techniques).

As we have a large amount of data, would it be a good solution to draw a random subsample from this dataset following the probabilities observed in the population by different characteristics?

$\endgroup$
4
  • 1
    $\begingroup$ What exactly does it mean that the "deviations are hard in some cases"? $\endgroup$ Commented Jul 22, 2012 at 19:39
  • 1
    $\begingroup$ I think we need a bit more information to answer your question. For example, what are the main of the "various reasons" it doesn't follow the original plan. Also, exactly what do you mean by calibrate - this has a narrow technical term in survey analysis, but your use suggests that perhaps you might mean post-stratification weighting to population. $\endgroup$ Commented Jul 22, 2012 at 20:06
  • $\begingroup$ Thanks for your comments. I'll try to explain better. At first the design try to get results for small regions and not all of them have been participated in the study. If we consider bigger areas we have data from all of them but the distribution in the sample is very different from the population (even for post-calibration techniques). Could I extract a subsample in these circumstances? $\endgroup$
    – Adolfo
    Commented Jul 22, 2012 at 20:46
  • $\begingroup$ Thanks @Adolfo - I've had a go at answering, and also edited your question to include the information your comment. Have a check that I haven't scrambled your meaning. $\endgroup$ Commented Jul 23, 2012 at 7:31

1 Answer 1

2
$\begingroup$

There are situations in survey analysis when you do a random resample similar to what you describe, with the intent of creating a new sample that can be treated as though it is a simple random sample. For example, this is sometimes done so the new sample can be plotted with each point having equal weight.

However, for general purposes this is less useful than keeping the full sample that you have gathered and using post-stratification weighting on it. If you know enough about the probabilities to resample in accordance with the population distribution, you know enough to give weights to the points in your original sample. This retains more of the information in your original sample but allows the overrepresented regions to be downweighted accordingly.

Basically, what I am suggesting is that though what you suggest is a possible solution, if you can implement it you can also implement a traditional weighting scheme, and retain your full sample. You will then need to analyse it using the usual techniques for complex surveys rather than for simple random sampling; but you will have a better solution than your resampling approach.

I recommend Thomas Lumley Complex Surveys: a guide to analysis using R published by Wiley, even if you aren't using R.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.