Example of a stochastic process that is 1st and 2nd order stationary, but not strictly stationary (Round 2)

Question

This question follows from my previous question, where Robin answered the question in the case of weak stationary processes. Here, I am asking a similar question for (strong?) stationary processes. I'll define what this means (the definition can also be found here).

Let $X(t)$ be a stochastic process. We say that $X(t)$ is Nth-order stationary if, for every $t_1, t_2, \dots, t_N$ we have that the joint cumulative density functions $$F_{X(t_1),X(t_2),\dots,X(t_N)} = F_{X(t_1 + \tau),X(t_2 + \tau),\dots,X(t_N + \tau)}$$ for all $\tau$.

This is quite a strong condition, it says that the joint statistics don't change at all as time shifts.

For example, a 1st order stationary process is such that $F_{X(t_1)} = F_{X(t_2)}$ for all $t_1$ and $t_2$. That is, the $X(t)$ are all identically distributed. It is quite easy to see that a 1st order stationary process need not be 2nd order stationary. Simply assign a correlation structure to say $X(t)$, $X(t+1)$, $X(t+2)$ that does not correspond to a (symmetric) Toeplitz matrix. That is, in vector form, the covariance matrix of $[ X(t), X(t+1), X(t+2)]$ could be given as

$$\left[\begin{array}{cc} \sigma^2 & a & b \newline a & \sigma^2 & c \newline b & c& \sigma^2 \end{array}\right]$$

for $a,b,c$ distinct. This is now not 2nd order stationary because $E[X(t)X(t+1)] = a$ and, time shifting by 1 we have $E[X(t+1)X(t+2)] = c \neq a$.

In a similar way (presumably), a process that is 1st and 2nd order stationary need not be 3rd order stationary and this leads to my question:

Does somebody have a nice example of a stochastic process that is both 1st and 2nd order stationary, but not 3rd order stationary?

Answer 1

Here is such an example. I will describe the process in terms of its sample paths.

It's a simple process, it only has four sample paths: $\omega_1$, $\omega_2$, $\omega_3$, $\omega_4$. This means that the only possible outcomes are:

$$ ...,\ X(-1) = \omega_i(-1),\ X(0) = \omega_i(0),\ X(1) = \omega_i(1),\ ... $$

One outcome for every $i \in \{1,2,3,4\}$.

The sample paths are defined as follows, for $t\in \mathbb{Z}$:

$$\omega_1(t):= \begin{cases} 1 & \text{if $t$ is a multiple of $4$,}\\ 0 & \text{else.}\\ \end{cases} $$

$$\omega_2(t):= \begin{cases} 0 & \text{if $t$ is a multiple of $4$,}\\ 1 & \text{else.}\\ \end{cases} $$

$$\omega_3(t):= \omega_1(t-2) $$

$$\omega_4(t):= \omega_2(t-2) $$

So they are:

$$ ...,1,0,0,0,1,0,0,0,1,0,0,0,1,0,0,0,... $$ $$ ...,0,1,1,1,0,1,1,1,0,1,1,1,0,1,1,1,... $$ $$ ...,0,0,1,0,0,0,1,0,0,0,1,0,0,0,1,0,... $$ $$ ...,1,1,0,1,1,1,0,1,1,1,0,1,1,1,0,1,... $$

Here is a plot of the four possible sample paths:

Each sample path is defined to have probability $1/4$. This concludes the definition of the process.

I will now show that it satisfies the conditions required by the OP.

For $t\in\{1,2,3,4\}$ one can see that there are exactly $2$ sample paths such that $X(t)=1$, and exactly $2$ sample paths such that $X(t)=0$. Hence for $t\in\{1,2,3,4\}$:

$$ P[X(t)=1] = P[X(t)=0] = 1/2 .$$

Since the process is periodic this is true for every $t\in \mathbb{Z}$. This means that the distribution of $X(t)$ does not depend on $t$ and therefore the process is 1st order stationary.

Likewise, looking at consecutive pairs, for $t\in\{1,2,3,4\}$ there is exactly $1$ sample path with $X(t)=1$ and $X(t+1)=0$, exactly $1$ sample path with $X(t)=0$ and $X(t+1)=1$, exactly $1$ sample path with $X(t)=0$ and $X(t+1)=0$, and exactly $1$ sample path with $X(t)=1$ and $X(t+1)=1$. So for $t\in\{1,2,3,4\}$:

$$ P[X(t)=1,X(t+1)=0] = P[X(t)=0,X(t+1)=1] =P[X(t)=0,X(t+1)=0] =P[X(t)=1,X(t+1)=1] = 1/4 .$$

Again by periodicity, this is true for every $t\in \mathbb{Z}$ and the process is 2nd order stationary.

However, for instance: $$P[X(0)=0,X(1)=0,X(2)=0] = 0,$$ while $$P[X(1)=0,X(2)=0,X(3)=0] = 1/4.$$

Therefore the process is not 3rd order stationary