It is well known that exchangeability refers to the following property $p(X_1,\dots, X_n) = p(X_{\pi(1)}, \dots, X_{\pi(n)})$ for any finite $n$ and a permutation $\pi$ when we have an infinite sequence of random variables $X_1, X_2, \dots$. So, for instance, when $n=2$, I can write that $p(X_1=1, X_2=0) = p(X_1=0, X_2=1)$.

Gaussian processes are also exchangeable. Let's say we have two varibles $X_1$ and $X_2$ at time $t=1$ and $t=2$ respectively. A zero-mean GP with a kernel $k$ gives the following density:

$p(X_1, X_2) = \mathcal N(\mathbf 0, \begin{bmatrix} k(X_1,X_1) & k(X_1,X_2) \\ k(X_2,X_1) & k(X_2,X_2) \end{bmatrix})$

or, if the order is permuted:

$p(X_2, X_1) = \mathcal N(\mathbf 0, \begin{bmatrix} k(X_2,X_2) & k(X_2,X_1) \\ k(X_1,X_2) & k(X_1,X_1) \end{bmatrix})$

In this case $p(X_1=1, X_2=0) \ne p(X_1=0, X_2=1)$, so it must be wrong to think about exchangeability in GPs this way. What is the correct way of thinking then?

  • $\begingroup$ Under a stationary gp (where the kernel only depends on the distance between $X$ and $Y$) $k(X,X) = \sigma^2$ for all $X$ and $k(X,Y) = k(Y,X)$. If you plug values into a stationary kernel you will see the covariance matrix will be the same regardless of the ordering of the $X_i$ $\endgroup$ – jcken Jun 14 at 16:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.