# Weak stationary time series implications

The time series $$\{X_t\}$$ is said to be weak stationary if for all $$n > ≥ 1$$ for any $$t_1, . . . ,t_n ∈ T$$ and for all $$τ$$ such that $$t_1+\tau, \dots ,t_n + τ ∈ T$$ all the joint moments of order 1 and 2 of $$X_{t_1} , . . . , X_{t_n}$$ exist, are all finite and equal to the corresponding joint moments of $$X_{t_1+\tau} , . . . , X_{t_n+\tau}$$ . In fact this corresponds to $$\Bbb E\{X_t\} = µ, Var\{X_t\} = σ^2$$ and $$\Bbb E\{X_{t_1}X_{t_2} \} = \Bbb E\{X_{t_1+\tau}X_{t_2+\tau} \}$$. One may deduce from this that $$\Bbb E\{X_{t_1}X_{t_2} \}$$ is a function of $$t_2 − t_1$$ only.

I do not understand the last statements of the above. First, I never heard about joint moments, is is just $$\Bbb E\{( X_{t_i}\dots X_{t_n})^k\}$$ ? If yes, then why is it said that $$\Bbb E\{X_{t_1}X_{t_2} \} = \Bbb E\{X_{t_1+\tau}X_{t_2+\tau} \}$$ and not $$\Bbb E\{X_{t_1}\dots X_{t_n} \} = \Bbb E\{X_{t_1+\tau}\dots X_{t_n+\tau} \}$$ ? (i.e. why do they stop at $$t_2$$ ?). Finally, I do not understand how we deduce from this that $$\Bbb E\{X_{t_1}X_{t_2} \}$$ is a function of $$t_2 − t_1$$ only, since $$X_i$$ can depend on a lot of things, if $$X_i$$s are $$Ber(p)$$ random variables, then I do not see why would the quantity $$\Bbb E\{X_{t_1}X_{t_2} \}$$ not depend on $$p$$ and depend just on $$t_2 − t_1$$. Can someone explain ?

• What is the source of your quoted paragraph? Commented Feb 22, 2023 at 22:59
• My teacher's lecture notes @Zhanxiong Commented Feb 22, 2023 at 23:06
• The definition seems unnecessarily complicated and unclear (I haven't heard the term "joint moments" either). For a clear (and standard) definition of the second-order weak stationarity, you can check Definition Definition 1.3.2 in Time Series: Theory and Methods by Brockwell and Davis. Commented Feb 23, 2023 at 1:23
• Wide sense stationary is that the full, joint distribution is invariant to time shifts: you get the same joint distribution in year 1880 as in the year 2080. Weak sense stationary is that the mean and covariance are invariant to time shifts. Commented Feb 23, 2023 at 3:11

Imagine we have four times $$a$$, $$b$$, $$c$$ and $$d$$.

Let $$\{X_t\}$$ be a weakly stationary stochastic process. Imagine that $$\operatorname{E}[X_{t_1}X_{t_2}] = f(t_1,t_2)$$ but that $$f(t_1,t_2)$$ cannot be written as $$g(t_2 - t_1)$$, that is, $$f$$ depends on something more than the length of the time interval. Hence there exist times $$a$$, $$b$$, $$c$$, and $$d$$ such that:

• $$b - a = d - c \quad$$ (i.e. the intervals $$(a, b)$$ and $$(c, d)$$ are the same length) AND
• $$f(a, b) \neq f(c, d)$$ hence $$\operatorname{E}[X_aX_b] \neq \operatorname{E}[X_cX_d]$$

Now consider time offset $$\tau = c - a$$ hence:

$$a + \tau = c \quad \quad b + \tau = d$$

If we apply the definition of weak stationarity:

$$\operatorname{E}[X_aX_b] = \operatorname{E}[X_{a + \tau}X_{b + \tau}] = \operatorname{E}[X_{c}X_{d}]$$

But this contradicts $$\operatorname{E}[X_aX_b]\neq \operatorname{E}[X_{c}X_{d}]$$. Hence we conclude that $$f(t_1, t_2)$$ cannot depend on more than the time interval $$t_2 - t_1$$.

### In more colloquial English

• Let $$\mu(t) = \operatorname{E[X_t]}$$ be the unconditional mean at time $$t$$
• Let $$K(t, \tau) = \operatorname{Cov}(X_t, X_{t + \tau})$$ be the unconditional auto-covariance function.

What weak-sense stationary gives you is that $$\mu(t)$$ and $$K(t, \tau)$$ are the same for all time $$t$$. The unconditional first and second moments (central or non-central) are the same everywhere. The coin has the same unconditional expectation and auto-covariance unfunction on the 10th flip as the 10,000th flip.

If weak-sense stationarity, we can simplify and just write:

• $$\operatorname{E}[X_t] = \mu \quad$$ (no dependence on time $$t$$)
• $$\operatorname{Cov}[X_t, X_{t + \tau}] = K(\tau) \quad$$ (depends only on the offset $$\tau$$)

### Some hypothetical violations of weak-sense stationarity

On the other hand, if the coin GOT DAMAGED with each flip in such a way to make it more likely to land hands, then weak stationarity would be violated. Or imagine you're rolling a die that slowly had an edge rounded off so that it would stop landing on the number 6 hence $$\operatorname{X}[X_t^2]$$ declined with time. Or imagine a die that started out as fair but overtime, developed some weird damage so that it was more likely to roll whatever the last number that was rolled (hence auto-correlation increases with time).