Exchangeability and IID random variables

Question

Every IID sequence of random variables is considered to be exchangeable, i understand why its necessary for the random variables to be identically distributed to assume exchangeability, but why the need for independence, (or is there a need)?

In the context of the definition which loosely states that any permutation of the random variables has the same joint distribution, isn't it sufficient for the random variables to be identically distributed to be able to reorder them, or must both conditions be met?

Answer 1

I think, the word "identically distributed" is mostly misleading when not used to discuss independent random variables. Consider the following example: $$\begin{pmatrix}X_1 \\ X_2 \\ X_3\end{pmatrix} \sim \mathrm{N}\left(\begin{pmatrix}0 \\ 0 \\ 0\end{pmatrix},\begin{pmatrix}1 &0 & 0 \\ 0&1&0.1 \\ 0&0.1&1\end{pmatrix} \right)$$

The components of the vector $(X_1, X_2, X_3)^T$ are neither independent, nor exchangeable, but they are identically distributed: the marginal distributions are all standard normal: $X_i \sim \mathrm{N}(0,1)$, $i = 1,2,3$.

Next example: $$\begin{pmatrix}Y_1 \\ Y_2 \\ Y_3\end{pmatrix} \sim \mathrm{N}\left(\begin{pmatrix}0 \\ 0 \\ 0\end{pmatrix},\begin{pmatrix}1 &0.1 & 0.1 \\ 0.1&1&0.1 \\ 0.1&0.1&1\end{pmatrix} \right)$$

The components are now not independent but exchangeable. The marginal distributions are again identical, standard normal: $Y_i \sim \mathrm{N}(0,1)$, $i = 1,2,3$.

We have in the end the following implications:

$$ \text{i.i.d. } \Rightarrow \text{ exchangeability } \Rightarrow \text{marginals identical}.$$

The counterexamples above show, that the converse implications are all wrong.

Answer 2

To answer this question you need to understand the "representation theorem" for exchangeable sequences of random variables (first stated by de Finetti and extended by Hewitt and Savage). This (brilliant) theorem says that every sequence of exchangeable random variables can be considered as a sequence of conditionally IID random variables, with distribution equal to the limiting empirical distribution of the sequence. This means that every sequence of conditionally IID random variables is exchangeable and every sequence of exchangeable random variables is conditionally IID. Conditional independence does not imply marginal independence, and it is common for exchangeable sequences of random variables to be positively correlated (but they cannot be negatively correlated).

In regard to your question, this means that independence is not a requirement for exchangeability, but conditional independence is. Most sequences of exchangeable random variables are positively correlated, owing to the fact that conditional independence generally induces an information link between the random variables.

Answer 3

There is no need for exchangeable random variables to be independent. For instance, if the vector $X$ follows a multivariate t distribution with mean zero , identity matrix as a scale matrix, and q degrees of freedom, then it's components are exchangeable, uncorrelated, and identically distributed, but not independent. Of course, by the exchangeability theorem it's components are conditionally iid (conditionally on a Gamma(q/2,q/2) in fact), but they are not independent.