13

The idea's right--but there's a question of expressing it a little more rigorously. I will therefore focus on notation and on exposing the essence of the idea. Let's begin with the idea of exchangeability: A random variable $\mathbf X=(X_1, X_2, \ldots, X_n)$ is exchangeable when the distributions of the permuted variables $\mathbf{X}^\sigma=(X_{\sigma(...


12

I think, the word "identically distributed" is mostly misleading when not used to discuss independent random variables. Consider the following example: $$\begin{pmatrix}X_1 \\ X_2 \\ X_3\end{pmatrix} \sim \mathrm{N}\left(\begin{pmatrix}0 \\ 0 \\ 0\end{pmatrix},\begin{pmatrix}1 &0 & 0 \\ 0&1&0.1 \\ 0&0.1&1\end{pmatrix} \right)$$ The ...


9

Exchangeability doesn't apply to a distribution, but a sequence of random variables. From wikipedia: Formally, an exchangeable sequence of random variables is a finite or infinite sequence $X_1, X_2, X_3, ...$ of random variables such that for any finite permutation $\sigma$ of the indices $1, 2, 3, ...$, [...] the joint probability distribution of the ...


8

Exchangeability, loosely writing, means you can permute the indices of the random variables in the expression $F(x_1, \dots, x_n)$ without having the result of the probability calculation change. This means, basically, that you can put the observed value of, for example, $x_1$, in where $x_3$ is in the list of values and vice versa (or more complex ...


8

One situation in which exchangeability does not hold occurs when we're testing whether means of two groups are equal, but suspect variances may be unequal. To be specific, let's look at the following situation: x1 is a sample of size $n_1 = 10$ from a normal population with $\mu_1=100$ and $\sigma_2=10$ and x2 is a sample of size $n_2 = 50$ from a normal ...


7

Spherical symmetry is a special case of exchangeability. The invariance property for spherically symmetric sequences that you describe is indeed the standard definition of spherical symmetry e.g. see On Ali's Characterization of the Spherical Normal Distribution, Steven F. Arnold and James Lynch, Journal of the Royal Statistical Society. Series B (...


7

The theorem in question tells us that exchangeability is equivalent to being conditionally IID. Hence, in practice, data analysts consider the same things when deciding whether observations are exchangeable as when deciding whether they're (conditionally) independent. The basic approach is to treat as a covariate anything that might account for dependencies ...


6

This is described by Diaconis (1988; see also Diaconis and Freedman, 1980): In 1938 de Finetti broaden the concept of exchangeability. Consider first the special case with two observations $X_1,X_2,\dots;Y_1,Y_2,\dots$. The $X_i$ might represent binary outcomes for a group of men and $Y_i$ might represent binary outcomes for a group of women. If ...


6

A regression model gives predictions of the response conditional on predictor values; so there's no problem in applying a model fitted to one set of predictor values fixed by design to another set of predictor values, even if the latter are randomly sampled from a population. With an experimental design matrix $X$, the expectation & variance of the ...


6

The product does not have to be exchangeable. The following counterexample will show what can go wrong and why. We will specify the joint distributions $P_1$ of $(X_1,Y_1)$ and $P_2$ of $(X_2,Y_2)$ and assume each of these bivariate random variables is independent. Thus, the $X_i$ will be exchangeable provided they are identically distributed, and ...


6

The phrase is true because if they weren't dependent, you can't learn about unsampled values. That is, learning $X_1,...,X_n$ does not improve your prediction for $X_{n+1}$ no matter how large $n$ is. Mathematically "independence" is saying $$p(X_{n+1}=x|X_1,...,X_n,I)=p(X_{n+1}=x|I)$$ In your example the dice rolls are not independent, provided the prior ...


6

Another important case is tests for interaction. The null hypothesis of additivity does not imply exchangeability. In a linear, constant variance model you can permute residuals (Anderson, 2001), in generalised linear models it's more complicated


6

There are many, many situations where exchangeability of values in a sequence does not hold. One general scenario is when you have a time-series of values that are autocorrelated, so that values near each other in time are statistically related. For example, if we produce a random walk, the values in the random walk are not exchangeable, and this will be ...


5

Not a complete answer, but here are two quick checks. Both need to be fullfilled in order to have the columns exchangeable. That is, if you see that there are differences then the 3 columns cannot be exchanged: First check is of course whether the 3 columns have the same univariate distributions. Second check: Generate enough samples and produce a ternary ...


5

$\newcommand{\one}{\mathbf 1}$This is not a proof (and +1 to @whuber's answer), but it's a geometric way to build some intuition as to why $E(X_1 | T) = T/n$ is a sensible answer. Let $X = (X_1,\dots,X_n)^T$ and $\one = (1,\dots,1)^T$ so $T = \one^TX$. We're then conditioning on the event that $\one^TX = t$ for some $t \in \mathbb R$, so this is like ...


4

De Finetti's theorem proves the existence of a prior distribution $H(\theta)$ such that $$ \int^1_0\theta\,^k(1-\theta)^{n-k}dH(\theta)=\frac{\Gamma(r+w)}{\Gamma(r)\Gamma(w)}\frac{\Gamma(r+k)\Gamma(w+n-k)}{\Gamma(r+w+n)} $$ Looking more closely at the proof of this theorem, it seems that the prior distribution is obtained as the solution to a Hausdorff ...


4

Exchangeability is not necessary. There are Bayesian models in which observations are not exchangeable. For example, models for time-series analysis and forecast in weather prediction or finance. Generally speaking, in such models more recent observations are considered to be more relevant for inference about future ones; a sort of "fading memory". ...


4

This is not possible in the general case. Assume the $a_i$s are unique constants. Exchangeability of the $X_i$s implies that $P(X_i = a_0)$ does not depend on $i$. That is, $P(X_i = a_0)=c$ where $c$ does not depend on $i$. With the construction defined in the question, we have \begin{equation} c = P(X_i = a_0) = P(T(i) = 0) = P(T^{-1}(0) = i). \end{...


4

I'm a bit late and good things have already been written, however I didn't see the following mentioned: For person A coin tosses are independent and identically distributed, hence they are exchangeable. For person B coin tosses are neither independent nor identically distributed but they are exchangeable." This doesn't depend on the information that A ...


4

Something that might be helpful in unifying these two views is the Hewitt-Savage(-de Finetti) representation theorem. The theorem says that $X_1,\dots,X_n$ are exchangeable precisely when they are independent and identically distributed conditional on some additional information. This is important in Bayesian statistics, because it means that an ...


3

Let $X=(X_1, \dotsc,X_n)$ be an exchangeable random vector. Then, specifically, its variance-covariance matrix (if it exists) must be (proportional to) $$ \Sigma = \begin{pmatrix} 1 & \rho & \dotsm & \rho \\ \rho & 1 & \dotsm & \rho \\ \ddots \\ ...


3

To answer this question you need to understand the "representation theorem" for exchangeable sequences of random variables (first stated by de Finetti and extended by Hewitt and Savage). This (brilliant) theorem says that every sequence of exchangeable random variables can be considered as a sequence of conditionally IID random variables, with distribution ...


3

A permutation test will still be valid under exchangeability rather than independence. In this case I think it is exchangeability of $+$ and $-$ signs across the observations (the set of ranks if your test statistic is the usual signed rank one) that matters. When performing the usual rank-based test it is common to assume that the pair-differences, $d_i=x_{...


3

On the face of it, your question is very general, so I doubt the answer can be simply a YES. So when can we use the de Finetti theorem? It is about (infinitely) exchangeable random variables, so for some counterexample we should ask if there are statistical models where exchangeability cannot be used. One obvious example is time series, where exchangeability ...


3

I will give a hint. The key concept is exchangeability, meaning that the random vector $(X_1, \dotsc, X_n)$ has the same distribution as $(X_{\pi 1}, \dotsc, X_{\pi n})$ for all permutations $\pi$ of $(1,2,\dotsc, n)$. Then you can check that the vector of ranks $(R_1, \dotsc, R_n)$ also will be exchangeable. Exchangeability is a generalization of iid, so ...


2

Schervish seems to use "distribution" to mean "induced probability measure", so I guess what is meant is that the probability measures of $X$ and $Y$ are equal, or equivalently, their CDFs are equal.


2

Exchangeability is generally tested by permutation tests (e.g., runs tests) which look at the number of "runs" in the sequence and compare it to its distribution under exchangeability. Remember that under the assumption of exchangeability, all $n!$ permutations of the $n$ observed values are equally probable, and so we can use this fact to simulate the ...


2

Equicorrelated random variables are another example for exchangeability but non-iid-ness. To take the simplest useful example, consider $n=3$. The correlation matrix then is $$ \begin{pmatrix} 1&\rho&\rho\\ \rho&1&\rho\\ \rho&\rho&1 \end{pmatrix}$$ As each r.v. has correlation $\rho$ (within the legitimate limits, see here) with any ...


2

The concept of degrees of freedom only applies to linear models, not to arbitrary random variables. This is because the degrees of freedom are defined as the dimension of a vector space. So, your question doesn't really make sense.


Only top voted, non community-wiki answers of a minimum length are eligible