Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

My spouse frequently works with (expensive, hard to obtain) data samples; for example route information for commuting bicyclists collected using a smartphone app. More often than not, these samples suffer from some kind of known demographic over-representation that they'd like correct for various applications.

Mindful of Karl Roves' and friends "corrections" to "obvious" democrat oversampling for the Nov. 2012 election polls which led to rather embarrassingly incorrect predictions, is there any theoretically appropriate way of doing this?

I'm not even sure what to call what I'm looking for -- is this what in some places is called reject inference?

share|improve this question
Try googling 'survey design weights'. I think those are what you want. – conjugateprior Nov 14 '12 at 21:52
In general "weighting" is what you're looking for... but in the example you give, weighting may be insufficient to the task. Weighting corrects for over- and under-representation, but it won't correct for relevant categories of people being totally excluded, for example because they don't own a smartphone. – Jonathan Nov 15 '12 at 20:08

This is the fundamental point of weighting a sample to population. You weight each individual in your sample based on known demographic features of the population such that the weights of each demographic group in the sample add up to population totals.

See any book on sampling theory and practice - no, it's not reject inference.

I recommend Thomas Lumley's survey package in R and the accompanying book on complex surveys. Even if you don't use R it is a great, clear introduction.

share|improve this answer
Many thanks. I do use R extensively, but will also check out the book. – pgoetz Nov 14 '12 at 23:01

The answer depends on if the data is collected by probabilistic sampling or not.

If this is a case of probabilistic sampling, then there are many good books in the field of survey sampling that could help you. The best to start with would be Lohr (2010) Sampling: Design and Analysis, 2nd Edition or Särndal, Swensson, Wretman (1992) Model Assisted Survey Sampling.

If data is not collected by probabilistic sampling (the inclusion probabilities of units in sample are not known) this is another case. I have a feeling from your description this could be your case. Propensity score weighting could be the tool for you (it is used in web surveys). There are papers by D.B. Rubin and others - for example paper by Lee (2009).

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.