17
$\begingroup$

I'm teaching an intro stats class and was reviewing the types of sampling, including systematic sampling where you sample every kth individual or object.

A student asked if sampling every person with a particular characteristic would accomplish the same thing.

For example, would sampling every person with a blue t-shirt be random enough and provide enough of a representation of the whole population? At least, if you're asking a question other than "What color t-shirt do you prefer wearing?" My sense is no, but I wondered if anyone here had any thoughts on this.

$\endgroup$
5
  • 12
    $\begingroup$ No. If you lived in Glasgow, Scotland, then most people who wear a "blue t-shirt", would probably be a Rangers supporter. You would be missing out on Celtic supporters. In Glasgow the football team would be a proxy for religion. $\endgroup$ Commented Feb 6, 2011 at 13:28
  • 1
    $\begingroup$ @csgillespie Wonderful example! $\endgroup$
    – whuber
    Commented Mar 6, 2011 at 23:45
  • $\begingroup$ Also, you might get more boys than girls because in western culture blue is associated with male $\endgroup$ Commented Sep 18, 2011 at 14:12
  • $\begingroup$ Colored T-shirts are more expensive than white T-shirts, and not all designs work for all colors. So, even though it might seem innocent, even among T-shirt wearers you may be selecting wealthier people, or more impulsive consumers, or people who favor a particular political party. $\endgroup$ Commented Feb 19, 2013 at 23:23
  • $\begingroup$ There's also an association between age and the wearing of t-shirts, between cultural background and wearing of t-shirts and so on. $\endgroup$
    – Glen_b
    Commented Mar 8, 2013 at 1:43

2 Answers 2

22
$\begingroup$

The answer, in general, to your question is "no". Obtaining a random sample from a population (especially of humans) is notoriously difficult. By conditioning on a particular characteristic, you're by definition not obtaining a random sample. How much bias this introduces is another matter altogether.

As a slightly absurd example, you wouldn't want to sample this way at, say, a football game between the Bears and the Packers, even if your population was "football fans". (Bears fans may have different characteristics than other football fans, even when the quantity you are interested in may not seem directly related to football.)

There are many famous examples of hidden bias resulting from obtaining samples in this way. For example, in recent US elections in which phone polls have been conducted, it is believed that people owning only a cell phone and no landline are (perhaps dramatically) underrepresented in the sample. Since these people also tend to be, by and large, younger than those with landlines, a biased sample is obtained. Furthermore, younger people have very different political beliefs than older populations. So, this is a simple example of a case where, even when the sample was not intentionally conditioned on a particular characteristic, it still happened that way. And, even though the poll had nothing to do with the conditioning characteristic either (i.e., whether or not one uses a landline), the effect of the conditioning characteristic on the poll's conclusions was significant, both statistically and practically.

$\endgroup$
6
$\begingroup$

As long as the distribution of the characteristic you are using to select units into the sample is orthogonal to the distribution of the characteristic of the population you want to estimate, you can obtain an unbiased estimate of the population quantity by conditioning selection on it. The sample is not strictly a random sample. But people tend to overlook that random samples are good because the random variable used to select units into sample is orthogonal to the distribution of the population characteristic, not because it is random.

Just think about drawing randomly from a Bernoulli with P(invlogit(x_i)) where x_i in [-inf, inf] is a feature of unit i such that Cov(x, y)!=0, and y is the population characteristic whose mean you want to estimate. The sample is "random" in the sense that you are randomizing before selecting into sample. But the sample does not yield an unbiased estimate of the population mean of y.

What you need is conditioning selection into sample on a variable that is as good as randomly assigned. I.e., that is orthogonal to the variable on which the quantity of interest depends. Randomizing is good because it insures orthogonality, not because of randomization itself.

$\endgroup$
1
  • 4
    $\begingroup$ This is correct, but how would you know if it was orthogonal unless you had a truly random sample? $\endgroup$
    – Peter Flom
    Commented Feb 6, 2011 at 15:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.