Every IID sequence of random variables is considered to be exchangeable, i understand why its necessary for the random variables to be identically distributed to assume exchangeability, but why the need for independence, (or is there a need)?

In the context of the definition which loosely states that any permutation of the random variables has the same joint distribution, isn't it sufficient for the random variables to be identically distributed to be able to reorder them, or must both conditions be met?

  • 2
    $\begingroup$ You might be confusing the joint distribution with the collection of marginal distributions. The former specifies much more than the latter. $\endgroup$ – whuber May 6 '18 at 19:29
  • 3
    $\begingroup$ "Every IID sequence of random variables is exchangeable" does not imply that "every exchangeable sequence of random variables must be IID." Consider a sequence of 10 variables generated as follows: we flip a coin. If it comes up heads, all 10 variables are set equal to 1, otherwise all 10 variables are set equal to 0. This is an exchangeable sequence but obviously not independent. $\endgroup$ – jbowman May 6 '18 at 21:07

I think, the word "identically distributed" is mostly misleading when not used to discuss independent random variables. Consider the following example: $$\begin{pmatrix}X_1 \\ X_2 \\ X_3\end{pmatrix} \sim \mathrm{N}\left(\begin{pmatrix}0 \\ 0 \\ 0\end{pmatrix},\begin{pmatrix}1 &0 & 0 \\ 0&1&0.1 \\ 0&0.1&1\end{pmatrix} \right)$$

The components of the vector $(X_1, X_2, X_3)^T$ are neither independent, nor exchangeable, but they are identically distributed: the marginal distributions are all standard normal: $X_i \sim \mathrm{N}(0,1)$, $i = 1,2,3$.

Next example: $$\begin{pmatrix}Y_1 \\ Y_2 \\ Y_3\end{pmatrix} \sim \mathrm{N}\left(\begin{pmatrix}0 \\ 0 \\ 0\end{pmatrix},\begin{pmatrix}1 &0.1 & 0.1 \\ 0.1&1&0.1 \\ 0.1&0.1&1\end{pmatrix} \right)$$

The components are now not independent but exchangeable. The marginal distributions are again identical, standard normal: $Y_i \sim \mathrm{N}(0,1)$, $i = 1,2,3$.

We have in the end the following implications:

$$ \text{i.i.d. } \Rightarrow \text{ exchangeability } \Rightarrow \text{marginals identical}.$$

The counterexamples above show, that the converse implications are all wrong.

  • 2
    $\begingroup$ +1: This answer is especially nice in providing clear, simple examples to demonstrate the assertions about the converse implications. Welcome to our site! $\endgroup$ – whuber May 6 '18 at 22:19

To answer this question you need to understand the "representation theorem" for exchangeable sequences of random variables (first stated by de Finetti and extended by Hewitt and Savage). This (brilliant) theorem says that every sequence of exchangeable random variables can be considered as a sequence of conditionally IID random variables, with distribution equal to the limiting empirical distribution of the sequence. This means that every sequence of conditionally IID random variables is exchangeable and every sequence of exchangeable random variables is conditionally IID. Conditional independence does not imply marginal independence, and it is common for exchangeable random variables to be positively correlated (but they cannot be negatively correlated).

In regard to your question, this means that independence is not a requirement for exchangeability, but conditional independence is. Most sequences of exchangeable random variables are positively correlated, owing to the fact that conditional independence generally induces an information link between the random variables.

  • $\begingroup$ Negative correlation is possible for variables that are only permutable or finitely exchangeable. Infinite exchangeability needed for the rep theorem, does not admit negative correlation $\endgroup$ – kjetil b halvorsen Dec 10 '18 at 21:23
  • 1
    $\begingroup$ Yes, that is correct. The classic example of finite exchangeability is SRSWOR, which gives a hypergeometric distribution. In that case there is negative correlation between the outcomes. $\endgroup$ – Ben Dec 10 '18 at 22:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.