# Time series, linear process, white noise

Let $$X_n = \sum\limits_{j=0}^{\infty} a_jZ_{t-j}$$, where $$Z_t$$ is a (weak) white noise $$(0,\sigma^2)$$ and $$a_j \in L^2$$. Prove that ACF $$\gamma_X(h)\longrightarrow 0$$ as $$h\longrightarrow +\infty$$. So:

$$\gamma_X(h)=\text{Cov}(X_{t+h},X_t)=\text{Cov}(\sum\limits_{i=0}^{\infty} a_iZ_{t+h-i}, \sum\limits_{j=0}^{\infty}a_jZ_{t-j})=\lim\limits_{n,m\to\infty} \text{Cov}(\sum\limits_{i=0}^{n} a_iZ_{t+h-i}, \sum\limits_{j=0}^{m}a_jZ_{t-j})\overset{\textbf{if true: why?}}{=} \lim\limits_{n,m\to\infty}\sum\limits_{i=0}^{n}\sum\limits_{j=0}^{m} \text{Cov}(a_iZ_{t+h-i}, a_jZ_{t-j})$$

and if we have weak white noise, then only non-zero terms of $$\text{Cov}$$ above are these that satisfy $$t+h-i=t-j \longleftrightarrow h=i-j$$, but... how to proceed from that point?

• One way is to observe that for any $\epsilon\gt 0,$ there exists $N$ such that for all $h\ge N,$ $a_h^2 + a_{h+1}^2 + \cdots \lt \epsilon.$
– whuber
Commented Mar 3, 2021 at 14:26
• @whuber thanks, but is everything correct up to the point where I left this exercise? Commented Mar 3, 2021 at 15:02
• It looks fine to me: you approach this with appropriate rigor with the limits. But you have only begun the calculation: that last summation simplifies greatly because the $Z_{t+h-i}$ and $Z_{t-j}$ are independent--hence uncorrelated--when $i-h\ne j.$
– whuber
Commented Mar 3, 2021 at 15:04

In terms of the Kronecker delta $$\delta(i,i)=1,$$ $$\delta(i,j)=0$$ when $$i\ne j,$$ you are concerned about the double limit

\begin{aligned} \lim\limits_{n,m\to\infty} \operatorname{Cov}\left( a_iZ_{t+h-i}, \sum\limits_{j=0}^{m}a_jZ_{t-j}\right) &= \lim\limits_{n,m\to\infty}\sum\limits_{i=0}^{n} \sum\limits_{j=0}^{m}\operatorname{Cov}\left( a_iZ_{t+h-i}, a_jZ_{t-j}\right)\\ &= \lim\limits_{n,m\to\infty}\sum\limits_{i=0}^{n} \sum\limits_{j=0}^{m}a_ia_j\operatorname{Cov}\left( Z_{t+h-i}, Z_{t-j}\right)\\ &= \lim\limits_{n,m\to\infty}\sum\limits_{i=0}^{n} \sum\limits_{j=0}^{m}a_ia_j \delta(t+h-i, t-j)\sigma^2\\ &= \sigma^2\lim\limits_{n,m\to\infty}\sum\limits_{j=0}^{\min(n, m-h)} a_{j+h}a_j.\\ \end{aligned}

The Cauchy-Schwarz Inequality gives an upper bound for the size of this sum,

\begin{aligned} \left|\sum\limits_{j=0}^{\min(n, m-h)} a_{j+h}a_j\right| &\le \left(\sum\limits_{j=0}^{\min(n, m-h)}a^2_{j+h} \sum\limits_{j=0}^{\min(n, m-h)}a^2_j\right)^{1/2}\\ &\le \left(\sum\limits_{j=0}^{\infty}a^2_{j+h} \sum\limits_{j=0}^{\infty}a^2_j\right)^{1/2}\\ &=\left(\left[\sum\limits_{j=0}^{\infty}a^2_{j} - \sum\limits_{j=0}^{h-1}a^2_{j} \right]\sum\limits_{j=0}^{\infty}a^2_j\right)^{1/2}. \end{aligned}

"$$(a)\in L^2$$" means $$\sum\limits_{j=0}^{\infty}a^2_{j}$$ converges to a finite value $$||a||_2^2.$$ Thus, given any $$\epsilon\gt 0$$ there exists $$h(\epsilon)$$ such that $$\sum\limits_{j=h(\epsilon)}^{\infty}a^2_{j} \le \epsilon^2.$$ In these terms the upper bound is

$$\left(\left[\sum\limits_{j=0}^{\infty}a^2_{j} - \sum\limits_{j=0}^{h(\epsilon)-1}a^2_{j} \right]\sum\limits_{j=0}^{\infty}a^2_j\right)^{1/2} = \left(\sum\limits_{j=h(\epsilon)}^{\infty}a^2_{j} \sum\limits_{j=0}^{\infty}a^2_j\right)^{1/2}\le \left(\epsilon^2 ||a||_2^2\right)^{1/2} = \epsilon ||a||_2.$$

Consequently, $$\left|\gamma_X(h)\right|$$ is bounded above by $$\epsilon\sigma^2||a||_2.$$ The limit of $$|\gamma_X(h)|$$ must lie between $$0$$ and the least upper bound of all such values, also equal to $$0,$$ showing $$\gamma_X(h)$$ converges absolutely to $$0,$$ whence it converges to $$0,$$ QED.