Having read a little about exchangeability, I went back to thinking about the iid condition required for the central limit theorem. It struck me that if two random variables are drawn from an identical distribution, the occurrence of one event does not make it more or less probable for the occurrence of another event. I have no doubts that I am wrong and there is a reason why both independence and identical distribution are required. I just don't know why!
$\begingroup$
$\endgroup$
3
-
9$\begingroup$ Well, $X$ and ... $X$ are identically distributed. $\endgroup$– NRHCommented Nov 2, 2011 at 22:33
-
3$\begingroup$ There are various forms of the central limit theorem. For example, identical distribution is not required if the independent random variables are uniformly bounded and $\sum_{i=1}^\infty \sigma_i^2 = \infty$, and mild dependence is also permissible. See, for example, the Wikipedia page on the central limit theorem. $\endgroup$– Dilip SarwateCommented Nov 2, 2011 at 23:18
-
2$\begingroup$ Consider gluing together a red and a blue coin. Throw the glued coins. Let $X$ be the outcome of the red coin and $Y$ be the outcome of the blue coin. Then both $X$ and $Y$ are Bernoulli random variables, yet they are not independent. $\endgroup$– Jens WagemakerCommented Oct 7, 2019 at 8:21
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
If random variables are drawn from an identical distribution, why doesn't this guarantee that they are independent?
Since you don't specify how the random variables are drawn, the question has no meaning. It is the manner of drawing that is important. Consider a neoclassical example of an urn with one ball marked $0$ and one ball marked $1$. The random variable $X$ is the number on the ball drawn from the urn, and is a Bernoulli random variable with parameter $p = P\{X = 1\} = \frac{1}{2}$. Now let $X_1$ denote the number on the first ball drawn and $X_2$ the number on the second ball drawn.
- Case I: drawing with replacement There are 4 equally likely outcomes of the experiment and they can be written as $00, 01, 10, 11$. $X_1$ and $X_2$ are both Bernoulli$\left(\frac{1}{2}\right)$ random variables and are independent.
- Case II: drawing without replacement Now there are only two equally likely outcomes $01, 10$ but $X_1$ and $X_2$ clearly are Bernoulli$\left(\frac{1}{2}\right)$ random variables just as before, and just as clearly are not independent.
Thus, just getting random variables with identical distribution does not by any means guarantee that they are independent.
answered Nov 3, 2011 at 1:42
-
5$\begingroup$ Thanks very much for the answer. But I am still unclear about one thing. in case II, you say X1, X2 ~ Bernoulli(1/2). But once X1 is drawn, the value of the second draw is now fixed -- so isn't X2 no longer a Bernoulli(1/2) random variable? In which case, X1 and X2 are no longer identically distributed? $\endgroup$– HFCCommented Nov 3, 2011 at 20:19
-
4$\begingroup$ Think of the experiment being repeated over and over again, and each time, a student comes in late, never finds out what the experiment consists of or that a first ball has been drawn. He sees each day that the professor draws a ball from the urn, and on roughly half the days the ball bears the number $1$ and the other times it bears the number $0$. He estimates that the ball is equally likely to be numbered $0$ or $1$, right? In Case II, there are two equally likely outcomes $01$ and $10$ and if you look only at the second ball, it is equally likely to bear the number $0$ or $1$. $\endgroup$ Commented Nov 3, 2011 at 22:08
-
14$\begingroup$ @HFC Less whimsically and more formally, it is important to not confuse between conditional distributions and unconditional distributions. In Case II, the conditional distribution of $X_2$ given $X_1$ is degenerate, and is not the same as the (unconditional) distribution of $X_1$. But, by the law of total probability, $$P(X_2=1)=P(X_2=1\mid X_1=1)P(X_1=1)+P(X_2=1\mid X_1=0)P(X_1=0)=0\times\frac{1}{2}+1\times\frac{1}{2}=\frac{1}{2}=P(X_1=1)$$ and so the unconditional distribution of $X_2$ is the same as the (unconditional) distribution of $X_1$. $\endgroup$ Commented Nov 3, 2011 at 23:21