In central limit theorem (in its most basic flavor), , we say that we draw a large number of samples from an unknown distribution and each time we calculate the mean. Then, for a large sample size, the mean of all these sample observations will tend to be a normal distribution, regardless of the original population distribution.

Now, are these 'samples' drawn with replacement, or without?

  • $\begingroup$ Does it matter? $\endgroup$
    – Neznajka
    Commented Oct 5, 2015 at 21:17

1 Answer 1


With replacement. In the statement of the theorem, we draw $n$ random sample from a fixed distribution, independently. If we sample without replacement the distribution is constantly changing, which does not meet the requirements of the theorem.

Typically sampling without replacement deals with finite sample spaces, but I suppose it could apply to infinite spaces. My intuition is that in an infinite discrete scenario without replacement, the distribution of $$ \sqrt{n}\left(\sum_{i=1}^n \frac{x_i}{n} - \mu \right) $$ would converge in distribution to a normal distribution with mean 0 and variance less than $\sigma^2$.

  • 1
    $\begingroup$ If the support of the distribution is unbounded--as for a geometric or Poisson distribution, for instance--then I think that expression should not converge at all. $\endgroup$
    – whuber
    Commented Oct 6, 2015 at 0:31
  • $\begingroup$ @whuber great point! And quite easy to prove! $\endgroup$ Commented Oct 6, 2015 at 0:37
  • $\begingroup$ If the support is bounded, then do you believe it converges with tighter variance? $\endgroup$ Commented Oct 6, 2015 at 0:38
  • $\begingroup$ Apart from exceptional circumstances, I expect the expression to diverge whenever the support is uncountable, whether or not it is bounded. As an example, let the distribution give probability $1/2$ to $0$ and the rest of its probability to numbers in an interval $(1-\epsilon,1+\epsilon)$ for tiny $\epsilon \lt 1/2$. Suppose $\mu=1/2$. The parenthesized expression converges rapidly to a number in the interval $(1/2-\epsilon,1/2+\epsilon)$. Multiplying it by $\sqrt{n}$ causes that to diverge. $\endgroup$
    – whuber
    Commented Oct 6, 2015 at 0:47
  • $\begingroup$ @whuber good point. Sampling without replacement should be done in moderation. Almost never infinitely! $\endgroup$ Commented Oct 6, 2015 at 0:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.