5
$\begingroup$

I have a sample survey of a population whose distributions of certain characteristics are not identical to the distributions of the overall population. For example, the age of my respondents may be biased downward, or the incomes in my sample may be too high compared to the population (or my theoretical population distribution, if the actual population distribution is unknown).

I know that it is possible to calculate a weighting coefficient that can adjust the distribution of the sample to match the population distribution on one dimension, but is it possible to adjust for two dimensions (e.g. age and income)? If so, couldn't there be a situation where there is no solution (no single coefficient that will adjust the sample distribution to the population distribution on both dimensions?)

$\endgroup$

2 Answers 2

3
$\begingroup$

The answer to the question, bracketing for the moment whether it is a good idea, is yes, there are ways to do this. If you have information on the joint distribution of the population over all the variables of interest (e.g. from a census), you can "poststratify" to that joint distribution, in the case of all discrete variables, or use "calibration" for more general combinations of discrete and continuous variables. If you have only information on the marginal distribution and you are willing to believe that an assumption of independence over the dimensions won't introduce too much bias, then you can "rake" to the margins. You can combine post-stratification, calibration, and raking to attend to arbitrary combinations of population information. Use a software package to do it though (e.g. the "survey" package in R or the svy suite of commands in Stata), because the computations can be tricky. For accessible introductions to these concepts with worked examples in R and Stata, respectively, see the following two references:

Lee and Forthofer. Analyzing Complex Survey Data. Sage.

Lumley. Complex Surveys: A Guide to Analysis Using R. Wiley.

Now, @Mohit asked why you would want to do this. Indeed, you should think about it. Intuitively, it would seem that you are reducing the bias in estimating population parameters from your sample. But there are cases in which such weighting can exacerbate biases; search for "non-ignorable" missingness or sample selection problems.

$\endgroup$
2
  • $\begingroup$ and a good short discussion and with two excellent references on Paul Hippel's pages $\endgroup$ Jan 10, 2011 at 21:31
  • $\begingroup$ thanks very much, that's the answer I was looking for. It was more of a curiosity thing and having the answer on hand to be able to explain to someone if they asked me about it, but I agree with you and @Mohit $\endgroup$
    – hgcrpd
    Jan 11, 2011 at 9:39
1
$\begingroup$

Just asking for a clarification : why would you want to do this? As I understand it, to do this properly, you need to have some concrete benchmark via which you are quantifying the bias.

The most important thing is how appropriate is the benchmark chosen? (Are we standardizing to the wrong base?)

Even if we have the correct benchmark, is it true that the the benchmark is exactly true? (The issue of matching each variable to the benchmark)

If you ask me, just proceed with the sample, and quantify separately for each variable the apparent bias from your sample to the "benchmark" population. I'd assume that this would be a more informative and accurate picture.

$\endgroup$
1
  • $\begingroup$ Our current policy is to not weight, since I, like you, decided that the weights were rather arbitrary and were probably no better than the unweighted results. When we sample, we target a specific distribution using each segment's proportional size, but this is also using a relatively arbitrary target distribution, so we may need to change this policy as well. $\endgroup$
    – hgcrpd
    Jan 11, 2011 at 9:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.