calculating truncated infinite AR weights in practice for Arima model

Question

I am trying to figure out how to form the truncated infinite AR weights for a general time series process.

$(1 - \phi_1 B - \phi_2 B^2 - ... - \phi_p B^p)(1 - B)z_t = (1-\theta_1 B - ... - \theta_q B^q)a_t$

Where $\phi \text{ and } \theta$ are some constants, $z_t$ is a series of measurements, $a_t$ is noise, and $B$ is the backshift operator $B * z_t = z_{t-1}$.

Let

$\phi(B) = (1 - \phi_1 B - \phi_2 B^2 - ... - \phi_p B^p)\\ \theta(B) = (1-\theta_1 B - ... - \theta_q B^q)\\ \rho(B) = \phi(B) * (1-B)$

Then this model can also be represented as : $\rho(B)z_t = \theta(B)a_t$

It is known that these models, if stationary, can be represented by an infinite AR series: $$\Psi(B) = \frac{\theta(B)}{\rho(B)} = \frac{1-\theta B}{(1-\phi B)(1-B)} = \sum_0^\infty \psi_iB^i$$

But, $\Psi(B)$ does not converge if the difference operator $(1-B)$ is included in $\rho(B)$, and according to "Time Series Analysis, Forecasting and Control"(Box, Jenkins), this series is only valid if we use the "truncated form" of the model.

The truncated form consists of a sum of a homogeneous and complementary solution with respect to some time point $k$: $$ z_t = C_k(t-k) + I_k(t-k)\\ \rho(B)C_k(t-k)=0\\ \rho(B)I_k(t-k) = (1-\theta B)a_t\\ $$

Here, $C_k$ is the homogeneous solution, and $I_k$ is the complementary solution, where $I_k(t-k) = 0$ when $t\leq k$, and $k$ represents a value where the series is truncated, such as the first data point in some time series.

1. How would one calculate the complementary function in practice?

The book gives the formula $$ I_k(t-k) = a_t + \psi_1 a_{t-1} + ... + \psi_{t-(k+1)} a_{k+1}\\ I_k(s-k) = 0 \text{ if } s \leq k $$

I was thinking I should write out the equation: $$\rho(B)I_k(t-k) = \theta(B)a_{t-k} = (1-\theta_1 B - \theta_2 B^2 - ... - \theta_q B^q) a_{t-k}$$ Then, possibly calculate $I_k(t-k)$ for every data point with $t \geq k$, equate coefficients of the left and right hand side, and solve using least squares if I have more data points than unknowns?

2. How do I calculate the homogeneous solution in practice?

The book gives the formula

$$C_k(t-k) = G_0^{t-k}\sum_{j=0}^{d-1}A_j(t-k)^j + \sum_{j=1}^p D_j G_j^{t-k}\\ \text{where } \rho(B) = (1-G_1)(1-G_2)...(1-G_p)(1-G_0)^d$$

Here, $\rho(B)$ has been factored so that you can see it's roots $(G_i)$. This is a general formula given for a case when one factor of $\rho(B)$ repeats $d$ times

In this case, should I use a root solving algorithm to find the roots of $\rho(B)$? If so, would I then form a system of equations with 1 equation for each time $t$, and solve that system for the $A_j$ and $D_j$, probably using least squares since more equations than unknowns (1 for each data point)?

Answer 1

"...infinite AR weights..." should be infinite MA weights.

Any MA series $z_t = \sum_{i = 0} \psi_i a_{t - i}$, where $\sum |\psi_i| < \infty$, starting at initial time $k$ can be expressed in "truncated-form" $$ z_t = \sum_{i = 0}^{t- (k + 1)} \psi_i a_{t - i} \,+ \, E[z_t | z_k, z_{k-1}, z_{k-2}, \cdots], $$ for $t \geq k$.

For example, take the MA representation of an AR$(1)$ series $z_t = \sum_{i = 0} \phi^i a_{t - i}$. The truncated form is $$ z_t = \sum_{i = 0}^{t-(k+1)} \phi^i a_{t - i} + \underbrace{ \phi^{t-k} z_k }_\text{$E[z_t|z_k, z_{k-1}, \cdots]$}, \;\; t \geq k. $$

What you are asking for is the ARIMA variation of this. In your notation, $$ \sum_{i = 0}^{t- (k + 1)} \psi_i a_{t - i} = I_k(t-k),\, E[z_t | z_k, z_{k-1}, z_{k-2}, \cdots] = C_k(t-k). $$

In the stationary $I(0)$ case, both the MA$(\infty)$ and truncated representation make sense. In the non-stationary $I(1)$ case, the MA$(\infty)$ representation does not make sense; only the truncated form does.

For example, a unit root, i.e. ARIMA(0,1,0), process can be written in truncated form $$ z_t = \sum_{i = 0}^{t-(k+1)} a_{t - i} + z_k, \; t \geq k $$ but $z_t = \sum_{i = 0}^{\infty} a_{t - i}$ does not make sense.

General Procedure

Let $k \geq 0$ be given. To compute the truncated-form of an ARIMA$(p, 1, q)$ series $$ (1 - \phi_1 B - \phi_2 B^2 - ... - \phi_p B^p)(1 - B)z_t = (1-\theta_1 B - ... - \theta_q B^q)a_t, \;\; t \geq k, $$ starting at $k$, do the following.

Step 1 Compute $\psi_i, i = 0, 1, \cdots, t - k -1$.

Since this is exactly the same as the ARMA case, I'll merge the $(1-B)$ factor of the AR polynomial and write $$ (1 - \phi_1 B - \phi_2 B^2 - ... - \phi_p B^p)(1 - B) = 1 - \varphi_1 B - \varphi_2 B^2 - ... - \varphi_{p+1} B^{p+1} = \Phi(B). $$

The weights $\{\psi_i\}_{i \geq 0}$ are solutions to the difference equations

\begin{align*} \psi_0 &= 1 \\ \psi_1 - \varphi_1 \psi_0 &= \theta_1 \\ \psi_2 - \varphi_1 \psi_1 - \varphi_2 \psi_0 &= \theta_2 \\ \vdots \\ \psi_{p} - \varphi_1 \psi_{p-2} - \cdots \varphi_{p} \psi_0 &= \theta_p, \\ \Phi(B)\psi_i &= 0, \;\; \forall i \geq p. \\ \end{align*}

To compute finitely many (in this case, $t-k$ many) $\psi_i$'s, simply solve the first $p+1$ equations for initial value and iterate forward. This gives $I_k(t-k)$.

In general, the system $\Phi(B)\psi_i = 0, \forall i \geq p$ can be solved like any linear homogeneous system of difference equations of degree $p+1$. The general solution $\{ \psi_i \}_{i \geq 0}$ is a linear combination of terms corresponding to the roots of the AR polynomial $\Phi$: $$ G^{-i} P_{m-1} $$ where $G$ is a real root of the multiplicity $m$ and $P_{m-1}$ is a polynomial in $i$ of degree $m-1$.
For example, if $\Phi$ have $p+1$ distinct real roots $G_1, \cdots, G_{p+1}$, the general solution is $$ H(t) = c_1 G_1^{-t} + \cdots c_{p+1} G_{p+1}^{-t}. $$ You can look up the case of complex roots. The coefficients $c_1, \cdots, c_{p+1}$ are given by the initial conditions.

Step 2 Derive $E[z_t | z_k, z_{k-1}, z_{k-2}, \cdots]$.

Follow the standard way to do this for ARIMA models. In general, $E[z_t | z_k, z_{k-1}, z_{k-2}, \cdots]$ is a function $f( z_k, z_{k-1}, z_{k-2}, \cdots)$ of $ z_k, z_{k-1}, z_{k-2}, \cdots$. For example, for the series $z_t = \phi z_{t-1} + a_{t}$ (with no restrictions on $\phi$), $$ E[z_t | z_k, z_{k-1}, z_{k-2}, \cdots] = \phi^{t-k} z_k = f(z_k). $$

Step 3 Solve for $C_k(t-k)$.

$C_k(t-k)$ is a solution to $\Phi(B) x_t = 0$ with $x_k = f( z_k, z_{k-1}, z_{k-2}, \cdots)$. In other words, $C_k(t-k)$ is just the general homogeneous solution $H(t)$ with the coefficients $c_1, \cdots, c_{p+1}$ determined by $H(k) = f( z_k, z_{k-1}, z_{k-2}, \cdots)$.

Again the AR(1) is an immediate simple example. In this case, $H(t) = c \phi^{-t}$. So $$ H(k) = f( z_k, z_{k-1}, z_{k-2}, \cdots) $$ means $$ c \phi^{-k} = z_k, $$ i.e. $c = \phi^k z_k$.