2
$\begingroup$

In central limit theorem (in its most basic flavor), , we say that we draw a large number of samples from an unknown distribution and each time we calculate the mean. Then, for a large sample size, the mean of all these sample observations will tend to be a normal distribution, regardless of the original population distribution.

Now, are these 'samples' drawn with replacement, or without?

$\endgroup$
  • $\begingroup$ Does it matter? $\endgroup$ – Neznajka Oct 5 '15 at 21:17
5
$\begingroup$

With replacement. In the statement of the theorem, we draw $n$ random sample from a fixed distribution, independently. If we sample without replacement the distribution is constantly changing, which does not meet the requirements of the theorem.

Typically sampling without replacement deals with finite sample spaces, but I suppose it could apply to infinite spaces. My intuition is that in an infinite discrete scenario without replacement, the distribution of $$ \sqrt{n}\left(\sum_{i=1}^n \frac{x_i}{n} - \mu \right) $$ would converge in distribution to a normal distribution with mean 0 and variance less than $\sigma^2$.

$\endgroup$
  • 1
    $\begingroup$ If the support of the distribution is unbounded--as for a geometric or Poisson distribution, for instance--then I think that expression should not converge at all. $\endgroup$ – whuber Oct 6 '15 at 0:31
  • $\begingroup$ @whuber great point! And quite easy to prove! $\endgroup$ – jlimahaverford Oct 6 '15 at 0:37
  • $\begingroup$ If the support is bounded, then do you believe it converges with tighter variance? $\endgroup$ – jlimahaverford Oct 6 '15 at 0:38
  • $\begingroup$ Apart from exceptional circumstances, I expect the expression to diverge whenever the support is uncountable, whether or not it is bounded. As an example, let the distribution give probability $1/2$ to $0$ and the rest of its probability to numbers in an interval $(1-\epsilon,1+\epsilon)$ for tiny $\epsilon \lt 1/2$. Suppose $\mu=1/2$. The parenthesized expression converges rapidly to a number in the interval $(1/2-\epsilon,1/2+\epsilon)$. Multiplying it by $\sqrt{n}$ causes that to diverge. $\endgroup$ – whuber Oct 6 '15 at 0:47
  • $\begingroup$ @whuber good point. Sampling without replacement should be done in moderation. Almost never infinitely! $\endgroup$ – jlimahaverford Oct 6 '15 at 0:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.