Having read a little about exchangeability, I went back to thinking about the iid condition required for the central limit theorem. It struck me that if two random variables are drawn from an identical distribution, the occurrence of one event does not make it more or less probable for the occurrence of another event. I have no doubts that I am wrong and there is a reason why both independence and identical distribution are required. I just don't know why!

  • 8
    $\begingroup$ Well, $X$ and ... $X$ are identically distributed. $\endgroup$
    – NRH
    Commented Nov 2, 2011 at 22:33
  • 2
    $\begingroup$ There are various forms of the central limit theorem. For example, identical distribution is not required if the independent random variables are uniformly bounded and $\sum_{i=1}^\infty \sigma_i^2 = \infty$, and mild dependence is also permissible. See, for example, the Wikipedia page on the central limit theorem. $\endgroup$ Commented Nov 2, 2011 at 23:18
  • 1
    $\begingroup$ Consider gluing together a red and a blue coin. Throw the glued coins. Let $X$ be the outcome of the red coin and $Y$ be the outcome of the blue coin. Then both $X$ and $Y$ are Bernoulli random variables, yet they are not independent. $\endgroup$ Commented Oct 7, 2019 at 8:21

1 Answer 1


If random variables are drawn from an identical distribution, why doesn't this guarantee that they are independent?

Since you don't specify how the random variables are drawn, the question has no meaning. It is the manner of drawing that is important. Consider a neoclassical example of an urn with one ball marked $0$ and one ball marked $1$. The random variable $X$ is the number on the ball drawn from the urn, and is a Bernoulli random variable with parameter $p = P\{X = 1\} = \frac{1}{2}$. Now let $X_1$ denote the number on the first ball drawn and $X_2$ the number on the second ball drawn.

  • Case I: drawing with replacement There are 4 equally likely outcomes of the experiment and they can be written as $00, 01, 10, 11$. $X_1$ and $X_2$ are both Bernoulli$\left(\frac{1}{2}\right)$ random variables and are independent.
  • Case II: drawing without replacement Now there are only two equally likely outcomes $01, 10$ but $X_1$ and $X_2$ clearly are Bernoulli$\left(\frac{1}{2}\right)$ random variables just as before, and just as clearly are not independent.

Thus, just getting random variables with identical distribution does not by any means guarantee that they are independent.

  • 5
    $\begingroup$ Thanks very much for the answer. But I am still unclear about one thing. in case II, you say X1, X2 ~ Bernoulli(1/2). But once X1 is drawn, the value of the second draw is now fixed -- so isn't X2 no longer a Bernoulli(1/2) random variable? In which case, X1 and X2 are no longer identically distributed? $\endgroup$
    – HFC
    Commented Nov 3, 2011 at 20:19
  • 4
    $\begingroup$ Think of the experiment being repeated over and over again, and each time, a student comes in late, never finds out what the experiment consists of or that a first ball has been drawn. He sees each day that the professor draws a ball from the urn, and on roughly half the days the ball bears the number $1$ and the other times it bears the number $0$. He estimates that the ball is equally likely to be numbered $0$ or $1$, right? In Case II, there are two equally likely outcomes $01$ and $10$ and if you look only at the second ball, it is equally likely to bear the number $0$ or $1$. $\endgroup$ Commented Nov 3, 2011 at 22:08
  • 14
    $\begingroup$ @HFC Less whimsically and more formally, it is important to not confuse between conditional distributions and unconditional distributions. In Case II, the conditional distribution of $X_2$ given $X_1$ is degenerate, and is not the same as the (unconditional) distribution of $X_1$. But, by the law of total probability, $$P(X_2=1)=P(X_2=1\mid X_1=1)P(X_1=1)+P(X_2=1\mid X_1=0)P(X_1=0)=0\times\frac{1}{2}+1\times\frac{1}{2}=\frac{1}{2}=P(X_1=1)$$ and so the unconditional distribution of $X_2$ is the same as the (unconditional) distribution of $X_1$. $\endgroup$ Commented Nov 3, 2011 at 23:21

Not the answer you're looking for? Browse other questions tagged or ask your own question.