# Mean square convergence of linear processes

In Brockwell and Davis's book (Introduction to time series and forecasting), a linear process is defined as

$X_t = \sum_{j=-\infty}^{\infty} \psi_jZ_{t-j}$

where $Z_{t} \sim WN(0, \sigma^2)$, $\psi_j$'s are constants such that $\sum_{-\infty}^{\infty}|\psi_j| < \infty$.

It states that $\sum_{-\infty}^{\infty}|\psi_j| < \infty$ ensures that the infinite sum converges (with probability one) as $E|Z_t| \leq \sigma < \infty$ and $E|X_{t}| < \infty$.

I am not sure how these three conditions help in proving almost sure convergence of the infinite sum.

Absolute summability will allow you to show that the sequence (in $$n$$) $$X_t^n = \sum_{j=-n}^{n} \psi_jZ_{t-j}$$ has a mean-square limit (we are not talking about almost-sure convergence, here). That is, we want to show that there exists some $$X_t$$ (we don't know it exists yet,because it's an infinite sum) such that $$\mathbb{E}[|X_t^n -X_t|^2] \to 0$$ as $$n \to \infty$$.

Often it is easier to verify that the sequence $$X_t^n$$ is Cauchy, which is an equivalent condition. This means that $$\mathbb{E}[|X_t^n -X_t^m|^2] \to 0$$ as $$m,n \to \infty$$.

Here's the proof. For $$n > m > 0$$ \begin{align*} &\mathbb{E}[|X_t^n -X_t^m|^2] \\ &= \mathbb{E}\left[\left| \sum_{m < |j| \le n} \psi_jZ_{t-j}\right|^2\right] \\ &= \sum_{m < |i| \le n} \sum_{m < |k| \le n} \psi_i \psi_k\mathbb{E}[Z_{t-i }Z_{t-k}] \\ &\le\sum_{m < |i| \le n} \sum_{m < |k| \le n} |\psi_i| |\psi_k| |\mathbb{E}[Z_{t-i}Z_{t-k}]| \tag{triangle ineq.}\\ &\le\sum_{m < |i| \le n} \sum_{m < |k| \le n} |\psi_i| |\psi_k| (\mathbb{E}[Z_{t-i}^2])^{1/2} (\mathbb{E}[Z_{t-k}^2])^{1/2} \tag{Cauchy-Schwarz}\\ &= \text{Var}(Z_t) \left( \sum_{m < |j| \le n} |\psi_j| \right)^2 \tag{stationarity of Z_t}\\ &\to 0 \tag{absolute summability}. \end{align*}

Only after you know that $$X_t$$ exists, can you show the order of taking the limit and expectation doesn't matter. Or in other words, you can show that

1. $$E[X_t^n] \to EX_t$$,
2. $$E[|X_t^n|^2] \to E[X_t^2]$$, and
3. $$E[X_t^nX_s^n] \to E[X_tX_s]$$;

but existence comes first.

## Edit:

To prove as convergence, we can use the Borel-Cantelli lemma. Pick $$\epsilon > 0$$ and call $$A_n = \{|X_t^n - X_t| > \epsilon \} = \left\{ \left|\sum_{|j|>n} \psi_j Z_{t-j}\right| > \epsilon\right\}.$$ Using the same reasoning above \begin{align*} \sum_{n=1}^{\infty} P(A_n) &\le \epsilon^{-2}\sum_{n=1}^{\infty} E\left[\left|\sum_{|j|>n} \psi_j Z_{t-j}\right|^2\right] \\ &\le \text{Var}(Z_t) \left( \sum_{j \in \mathbb{Z}} |\psi_j| \right)^2\\ &< \infty. \end{align*}

So $$X_t^n \overset{as}{\to} X_t$$ for each $$t$$.

• Nice, though OP asks about a.s. convergence. – Math-fun May 16 '19 at 7:14
• @Math-fun you're right. See edit – Taylor May 17 '19 at 17:00
• Thank you very much! +1 – Math-fun May 21 '19 at 7:53
• @Math-fun you are most welcome! – Taylor May 21 '19 at 14:25
• I looked at the three series theorem but then your proof is of course fine :-) – Math-fun May 21 '19 at 15:30