Mean square convergence of linear processes

Question

In Brockwell and Davis's book (Introduction to time series and forecasting), a linear process is defined as

$ X_t = \sum_{j=-\infty}^{\infty} \psi_jZ_{t-j}$

where $Z_{t} \sim WN(0, \sigma^2)$, $\psi_j$'s are constants such that $\sum_{-\infty}^{\infty}|\psi_j| < \infty$.

It states that $\sum_{-\infty}^{\infty}|\psi_j| < \infty$ ensures that the infinite sum converges (with probability one) as $E|Z_t| \leq \sigma < \infty$ and $E|X_{t}| < \infty$.

I am not sure how these three conditions help in proving almost sure convergence of the infinite sum.

Answer 1

Absolute summability will allow you to show that the sequence (in $n$) $$ X_t^n = \sum_{j=-n}^{n} \psi_jZ_{t-j} $$ has a mean-square limit (we are not talking about almost-sure convergence, here). That is, we want to show that there exists some $X_t$ (we don't know it exists yet,because it's an infinite sum) such that $$ \mathbb{E}[|X_t^n -X_t|^2] \to 0 $$ as $n \to \infty$.

Often it is easier to verify that the sequence $X_t^n$ is Cauchy, which is an equivalent condition. This means that $\mathbb{E}[|X_t^n -X_t^m|^2] \to 0$ as $m,n \to \infty$.

Here's the proof. For $n > m > 0$ \begin{align*} &\mathbb{E}[|X_t^n -X_t^m|^2] \\ &= \mathbb{E}\left[\left| \sum_{m < |j| \le n} \psi_jZ_{t-j}\right|^2\right] \\ &= \sum_{m < |i| \le n} \sum_{m < |k| \le n} \psi_i \psi_k\mathbb{E}[Z_{t-i }Z_{t-k}] \\ &\le\sum_{m < |i| \le n} \sum_{m < |k| \le n} |\psi_i| |\psi_k| |\mathbb{E}[Z_{t-i}Z_{t-k}]| \tag{triangle ineq.}\\ &\le\sum_{m < |i| \le n} \sum_{m < |k| \le n} |\psi_i| |\psi_k| (\mathbb{E}[Z_{t-i}^2])^{1/2} (\mathbb{E}[Z_{t-k}^2])^{1/2} \tag{Cauchy-Schwarz}\\ &= \text{Var}(Z_t) \left( \sum_{m < |j| \le n} |\psi_j| \right)^2 \tag{stationarity of $Z_t$}\\ &\to 0 \tag{absolute summability}. \end{align*}

Only after you know that $X_t$ exists, can you show the order of taking the limit and expectation doesn't matter. Or in other words, you can show that

1. $E[X_t^n] \to EX_t$,
2. $E[|X_t^n|^2] \to E[X_t^2]$, and
3. $E[X_t^nX_s^n] \to E[X_tX_s]$;

but existence comes first.

Edit:

To prove as convergence, we can use the Borel-Cantelli lemma. Pick $\epsilon > 0$ and call $$ A_n = \{|X_t^n - X_t| > \epsilon \} = \left\{ \left|\sum_{|j|>n} \psi_j Z_{t-j}\right| > \epsilon\right\}. $$ Using the same reasoning above \begin{align*} \sum_{n=1}^{\infty} P(A_n) &\le \epsilon^{-2}\sum_{n=1}^{\infty} E\left[\left|\sum_{|j|>n} \psi_j Z_{t-j}\right|^2\right] \\ &\le \text{Var}(Z_t) \left( \sum_{j \in \mathbb{Z}} |\psi_j| \right)^2\\ &< \infty. \end{align*}

So $X_t^n \overset{as}{\to} X_t$ for each $t$.