# Which probability distributions are not exchangeable?

Are there any specific families of probability distribution which are not exchangeable by construction?

I was thinking that the Hyper Geometric distribution would not be since it models random sampling without replacement, but I'm not sure?

• In its usual sense, "exchangeable" applies only to multivariate distributions. Evidently you are using this term differently. Please, then, explain what you understand "exchangeable" to mean. – whuber Aug 12 '16 at 13:59

Exchangeability doesn't apply to a distribution, but a sequence of random variables. From wikipedia:

Formally, an exchangeable sequence of random variables is a finite or infinite sequence $X_1, X_2, X_3, ...$ of random variables such that for any finite permutation $\sigma$ of the indices $1, 2, 3, ...$, [...] the joint probability distribution of the permuted sequence $X_{\sigma (1)},X_{\sigma (2)},X_{\sigma (3)},\dots$ is the same as the joint probability distribution of the original sequence.

A sequence $\textbf{k}$ of draws from a hypergeometric distribution has mass $P(\textbf{k}) = \prod_{i=0}^m P(k_i)$, which you can see is invariant to permutation, and thus exchangeable.

What I think you might be curious about is whether the sequence of successes summarized by a hypergeometric is exchangeable. It is: The PMF depends only on $k$, the count of successes, and not their order.

For examples of sequences that are non-exchangeable, any sequence $X_1, X_2, X_3, ...$ in which each $X_i$ depends solely on the prior random variable $X_{i-1}$ will do. (Meaning, $X_i \perp\mspace{-8mu}\perp X_{j \lt i-1} | X_{i-1}$)

Say, $X_i|x_{i-1} \sim N(x_{i-1},1)$, with $X_1 \sim N(0,1)$:

import numpy as np
from scipy.stats import norm

X = np.array([0,1,2,3])
X_permuted = np.random.permutation(X)
X_diff = np.diff(X)
X_permuted_diff = np.diff(X_permuted)

assert norm.pdf(X_diff).prod() == norm.pdf(X_diff_perm).prod()


You can also view this example as $X_i = x_{i-1} + \epsilon_i$ with $\epsilon_i \sim N(0,1)$, which provides a—helpful, I hope—counterexample: If you take the differences first, they're exchangeable.

# This won't throw an error
X_diff_permuted = np.random.permutation(X_diff)
assert norm.pdf(X_diff).prod() == norm.pdf(X_diff_permuted).prod()

• Seems correct, if the trivial possibility of the iid case is excluded. Note that this is the Markov property. (I'll delete my previous comments as they are now obsolete) – Juho Kokkala Aug 12 '16 at 18:47
• @JuhoKokkala Yes, "the Markov property" was what I intended and would have been much more succinct of me, but I didn't want to send the OP toward more technical terminology. Thanks again! – Sean Easter Aug 12 '16 at 19:25

I am not sure I completely understand your question. The term exchangeability refers to a sequence of random variables. A sequence of random variables is exchangeable when is invariant to random permutations. A sequence of random variable is exchangeable as long as the random variables are i.i.d.. Therefore if you have lets say some samples for any distribution that are independent one another then you have an exchangeable sequence of random variables.