Weak stationary time series implications

Question

The time series $\{X_t\}$ is said to be weak stationary if for all $n > ≥ 1$ for any $t_1, . . . ,t_n ∈ T$ and for all $τ$ such that $t_1+\tau, \dots ,t_n + τ ∈ T$ all the joint moments of order 1 and 2 of $X_{t_1} , . . . , X_{t_n}$ exist, are all finite and equal to the corresponding joint moments of $X_{t_1+\tau} , . . . , X_{t_n+\tau}$ . In fact this corresponds to $\Bbb E\{X_t\} = µ, Var\{X_t\} = σ^2$ and $\Bbb E\{X_{t_1}X_{t_2} \} = \Bbb E\{X_{t_1+\tau}X_{t_2+\tau} \}$. One may deduce from this that $\Bbb E\{X_{t_1}X_{t_2} \}$ is a function of $t_2 − t_1$ only.

I do not understand the last statements of the above. First, I never heard about joint moments, is is just $\Bbb E\{( X_{t_i}\dots X_{t_n})^k\}$ ? If yes, then why is it said that $\Bbb E\{X_{t_1}X_{t_2} \} = \Bbb E\{X_{t_1+\tau}X_{t_2+\tau} \}$ and not $\Bbb E\{X_{t_1}\dots X_{t_n} \} = \Bbb E\{X_{t_1+\tau}\dots X_{t_n+\tau} \}$ ? (i.e. why do they stop at $t_2$ ?). Finally, I do not understand how we deduce from this that $\Bbb E\{X_{t_1}X_{t_2} \}$ is a function of $t_2 − t_1$ only, since $X_i$ can depend on a lot of things, if $X_i$s are $Ber(p)$ random variables, then I do not see why would the quantity $\Bbb E\{X_{t_1}X_{t_2} \}$ not depend on $p$ and depend just on $t_2 − t_1$. Can someone explain ?

Answer 1

Imagine we have four times $a$, $b$, $c$ and $d$.

Let $\{X_t\}$ be a weakly stationary stochastic process. Imagine that $\operatorname{E}[X_{t_1}X_{t_2}] = f(t_1,t_2)$ but that $f(t_1,t_2)$ cannot be written as $g(t_2 - t_1)$, that is, $f$ depends on something more than the length of the time interval. Hence there exist times $a$, $b$, $c$, and $d$ such that:

• $b - a = d - c \quad $ (i.e. the intervals $(a, b)$ and $(c, d)$ are the same length) AND
• $f(a, b) \neq f(c, d)$ hence $\operatorname{E}[X_aX_b] \neq \operatorname{E}[X_cX_d]$

Now consider time offset $\tau = c - a$ hence:

$$ a + \tau = c \quad \quad b + \tau = d$$

If we apply the definition of weak stationarity:

$$ \operatorname{E}[X_aX_b] = \operatorname{E}[X_{a + \tau}X_{b + \tau}] = \operatorname{E}[X_{c}X_{d}] $$

But this contradicts $\operatorname{E}[X_aX_b]\neq \operatorname{E}[X_{c}X_{d}]$. Hence we conclude that $f(t_1, t_2)$ cannot depend on more than the time interval $t_2 - t_1$.

In more colloquial English

• Let $\mu(t) = \operatorname{E[X_t]}$ be the unconditional mean at time $t$
• Let $K(t, \tau) = \operatorname{Cov}(X_t, X_{t + \tau})$ be the unconditional auto-covariance function.

What weak-sense stationary gives you is that $\mu(t)$ and $K(t, \tau)$ are the same for all time $t$. The unconditional first and second moments (central or non-central) are the same everywhere. The coin has the same unconditional expectation and auto-covariance unfunction on the 10th flip as the 10,000th flip.

If weak-sense stationarity, we can simplify and just write:

• $\operatorname{E}[X_t] = \mu \quad $ (no dependence on time $t$)
• $\operatorname{Cov}[X_t, X_{t + \tau}] = K(\tau) \quad $ (depends only on the offset $\tau$)

Some hypothetical violations of weak-sense stationarity

On the other hand, if the coin GOT DAMAGED with each flip in such a way to make it more likely to land hands, then weak stationarity would be violated. Or imagine you're rolling a die that slowly had an edge rounded off so that it would stop landing on the number 6 hence $\operatorname{X}[X_t^2]$ declined with time. Or imagine a die that started out as fair but overtime, developed some weird damage so that it was more likely to roll whatever the last number that was rolled (hence auto-correlation increases with time).