5
$\begingroup$

When doing a simple random sample to estimate population mean for some statistic, how do I know whether sampling happens with or without replacement?

It feels wrong to use replacement, because 1) my AP stats teacher never does that and 2) I might use someone's data twice in the average.

But on the other hand, the proof that the statistic is an unbiased estimator of the mean is $$E(X)=E(X_1)+\cdots +E(X_n)=\mu+\cdots+\mu$$ which implies $$E\left(\frac{X}{n}\right)=\frac{n\mu}{n}=\mu$$ But doesn't this assume that the $n$ statistics $X_i$ are independent of each other? And isn't that only true if we replace after each sample?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Linearity of expectation doesn't rely on independence.

It's only the variance that's affected. If you sample without replacement (as most - but not all - population sampling is done), it reduces the variance a little (at least it's little under the common situation where the sample is much smaller than the population; hence the rule of thumb about ignoring it when the sample is sufficiently small)

For simple random sampling without replacement, it's actually quite easy to work out the mean and variance from fairly simple reasoning.

Formulas for estimating means and proportions under sampling without replacement are readily found - for example, here.

Improve this answer
$\endgroup$
11
  • $\begingroup$ Thanks! Is there ever a reason to sample without replacement, if I could sample with? What is the point of using sampling without replacement? $\endgroup$
    – user45031
    May 5, 2014 at 23:47
  • $\begingroup$ In some situations it's at least inconvenient to avoid the small risk of sampling the same unit twice. In most sampling, without replacement is quite natural, but under some conditions it's not. For example, depending on how the technology is set up, random digit dialing might sample with replacement, but the miniscule chance of dialing the same number twice (sample size relative to population being sampled) may well make it not worth worrying about (though one might guard against it by simply asking first if they'd already been surveyed). $\endgroup$
    – Glen_b
    May 5, 2014 at 23:52
  • $\begingroup$ What's wrong with using the same data twice though? As long as the random numbers told you to do so, you haven't done anything wrong right? Isn't "avoiding the risk" introducing human bias into the analysis? $\endgroup$
    – user45031
    May 6, 2014 at 0:26
  • $\begingroup$ I didn't say anything was wrong. You just asked why you wouldn't always sample without replacement. I gave an example where you might not; the cost is that you might have slightly higher variance. There's no "human bias" in either sampling with or without replacement; it's a choice you can make a priori, and if your technology gives you the second but you want the first it's not necessarily difficult to still work as if you had the second. $\endgroup$
    – Glen_b
    May 6, 2014 at 1:02
  • 1
    $\begingroup$ The gain of sampling WOR depends on the fraction of the population that is sampled. This may or not be "very little". Moreover, in multi-stage samples, if PSUs are sampled WR, then a different set of second stage units should be selected each time a PSU is drawn. $\endgroup$
    – Steve Samuels
    May 8, 2014 at 1:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.