10
$\begingroup$

Every IID sequence of random variables is considered to be exchangeable, i understand why its necessary for the random variables to be identically distributed to assume exchangeability, but why the need for independence, (or is there a need)?

In the context of the definition which loosely states that any permutation of the random variables has the same joint distribution, isn't it sufficient for the random variables to be identically distributed to be able to reorder them, or must both conditions be met?

$\endgroup$
2
  • 2
    $\begingroup$ You might be confusing the joint distribution with the collection of marginal distributions. The former specifies much more than the latter. $\endgroup$
    – whuber
    May 6, 2018 at 19:29
  • 4
    $\begingroup$ "Every IID sequence of random variables is exchangeable" does not imply that "every exchangeable sequence of random variables must be IID." Consider a sequence of 10 variables generated as follows: we flip a coin. If it comes up heads, all 10 variables are set equal to 1, otherwise all 10 variables are set equal to 0. This is an exchangeable sequence but obviously not independent. $\endgroup$
    – jbowman
    May 6, 2018 at 21:07

3 Answers 3

21
$\begingroup$

I think, the word "identically distributed" is mostly misleading when not used to discuss independent random variables. Consider the following example: $$\begin{pmatrix}X_1 \\ X_2 \\ X_3\end{pmatrix} \sim \mathrm{N}\left(\begin{pmatrix}0 \\ 0 \\ 0\end{pmatrix},\begin{pmatrix}1 &0 & 0 \\ 0&1&0.1 \\ 0&0.1&1\end{pmatrix} \right)$$

The components of the vector $(X_1, X_2, X_3)^T$ are neither independent, nor exchangeable, but they are identically distributed: the marginal distributions are all standard normal: $X_i \sim \mathrm{N}(0,1)$, $i = 1,2,3$.

Next example: $$\begin{pmatrix}Y_1 \\ Y_2 \\ Y_3\end{pmatrix} \sim \mathrm{N}\left(\begin{pmatrix}0 \\ 0 \\ 0\end{pmatrix},\begin{pmatrix}1 &0.1 & 0.1 \\ 0.1&1&0.1 \\ 0.1&0.1&1\end{pmatrix} \right)$$

The components are now not independent but exchangeable. The marginal distributions are again identical, standard normal: $Y_i \sim \mathrm{N}(0,1)$, $i = 1,2,3$.

We have in the end the following implications:

$$ \text{i.i.d. } \Rightarrow \text{ exchangeability } \Rightarrow \text{marginals identical}.$$

The counterexamples above show, that the converse implications are all wrong.

$\endgroup$
3
  • 3
    $\begingroup$ +1: This answer is especially nice in providing clear, simple examples to demonstrate the assertions about the converse implications. Welcome to our site! $\endgroup$
    – whuber
    May 6, 2018 at 22:19
  • 2
    $\begingroup$ I like the examples and how they demosntrate the implications you ascert, but i don't understand why "identically distributed is misleading when not discussing independent random variables" $\endgroup$ May 31, 2021 at 7:18
  • $\begingroup$ It is probably a personal thing, but "identically distributed" gives me a feeling of symmetry amongst the random variables. This symmetry though only exists for the marginals, but not within the random vector. Hence, my feeling misleads me. $\endgroup$
    – Jonas
    Jun 4, 2021 at 23:44
4
$\begingroup$

To answer this question you need to understand the "representation theorem" for exchangeable sequences of random variables (first stated by de Finetti and extended by Hewitt and Savage). This (brilliant) theorem says that every sequence of exchangeable random variables can be considered as a sequence of conditionally IID random variables, with distribution equal to the limiting empirical distribution of the sequence. This means that every sequence of conditionally IID random variables is exchangeable and every sequence of exchangeable random variables is conditionally IID. Conditional independence does not imply marginal independence, and it is common for exchangeable sequences of random variables to be positively correlated (but they cannot be negatively correlated).

In regard to your question, this means that independence is not a requirement for exchangeability, but conditional independence is. Most sequences of exchangeable random variables are positively correlated, owing to the fact that conditional independence generally induces an information link between the random variables.

$\endgroup$
4
  • 1
    $\begingroup$ Negative correlation is possible for variables that are only permutable or finitely exchangeable. Infinite exchangeability needed for the rep theorem, does not admit negative correlation $\endgroup$ Dec 10, 2018 at 21:23
  • 1
    $\begingroup$ Yes, that is correct. The classic example of finite exchangeability is SRSWOR, which gives a hypergeometric distribution. In that case there is negative correlation between the outcomes. $\endgroup$
    – Ben
    Dec 10, 2018 at 22:16
  • $\begingroup$ I think the second paragraph is incorrect. In the finite case with negatively correlation, conditional independence is clearly not a requirement. Is not even possible as any form of conditional independence would lead to non-negative correlation marginally. So conditional independence can only be a requirement for infinite exchangeability. $\endgroup$ May 31, 2021 at 9:17
  • $\begingroup$ @JarleTufto: Both the question and this answer refer to infinite sequences of random variables, not finite vectors of them, so I do not consider the finite case here. $\endgroup$
    – Ben
    May 31, 2021 at 9:26
3
$\begingroup$

There is no need for exchangeable random variables to be independent. For instance, if the vector $X$ follows a multivariate t distribution with mean zero , identity matrix as a scale matrix, and q degrees of freedom, then it's components are exchangeable, uncorrelated, and identically distributed, but not independent. Of course, by the exchangeability theorem it's components are conditionally iid (conditionally on a Gamma(q/2,q/2) in fact), but they are not independent.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.