Parameter space restriction in random walk + noise model

Question

Suppose we have a random walk + noise model so

\begin{align} y_t & = \mu_{t-1} + \epsilon_t\\ \mu_t & = \mu_{t-1} + \eta_t \end{align}

Then, it's straightforward to show that

$$\triangle y_t = \eta_{t-1} + \epsilon_t - \epsilon_{t-1}$$

and also that, the autocorrelation at lag 1 of $\Delta y_t$, $\gamma_1 = - \frac{\sigma^2_{\epsilon}}{\sigma^2_\eta + 2 \sigma^2_{e}}$

So, the lag 1 autocorrelation has to be between $-0.5$ and $0$ by inspection.

But, consider the "equivalent" ARIMA(0,1,1) model:

$$\triangle y_t = \theta \nu_{t-1} + \nu_t$$

Since, $\theta$ has to be between -1 and + 1 in the ARIMA(0,1,1) and the first order autocorrelation is $\frac{\theta}{1+\theta^2}$, the first order autocorrelation is between -0.5 and 0.5.

So, in the random walk + noise formulation, the $\theta$ is restricted to be between $-1$ and $0$ and, in the ARIMA(0,1,1) formulation, $\theta$ is between $-1$ and $1$.

So, my only point is that there is a restriction for the random walk + noise model. But, if we put $\phi$ as a coefficient of $\mu_t$ in the random walk + noise model so that it's no longer a random walk + noise model, then the restriction goes away.

So, by enforcing the unit root for $\mu_{t}$, one ends up with a restriction in the "equivalence" between the random walk + noise model and the ARIMA(0,1,1) model. In this sense, the two formulations are not truly equivalent because there are regions of the ARIMA(0,1,1) parameter space that cannot be reached by the supposed equivalent random walk + noise model.

Does this make intuitive sense to anyone that the restriction goes away by the introduction of the new parameter $\phi$. Just curious? Thanks.

Answer 1

Answer:

Every unit-root state-space model has an equivalent ARIMA(0,1,1) representation, but not every ARIMA(0,1,1) model has an equivalent state-space model representation.

The following holds true for the representations of unit-root state-space models:

1. Whatever the magnitude of the signal-to-noise ratio $\frac{\sigma_\eta^2}{\sigma_\epsilon^2}$ in the state-space model, MA weight $\theta$ of the MA term in the equivalent ARIMA representation lies in the interval $(-1,0)$.

Moreover, as the signal-to-noise ratio $\frac{\sigma_\eta^2}{\sigma_\epsilon^2}$ increases

2. The magnitude ($|\theta|$) of the delayed effect of the shocks in the equivalent representation decreases, and
3. The variance of the shock in the equivalent representation $\sigma_\nu^2$ must increase.

---

Discussion:

When talking about state-space models it is often convenient to introduce the signal-to-noise ratio, $\xi := \frac{\sigma^2_\eta}{\sigma^2_\epsilon}$. Let us try to see how this metric of the model affects the equivalent ARIMA representation.

ARIMA representation can be found by the method of undetermined coefficients. We know that $\Delta y_t$ has autocovariances

\begin{equation} Cov(\Delta y_t, \Delta y_{t-s}) = \begin{cases} (\xi + 2)\sigma_\epsilon^2 & \text{ for } s=0,\\ -\sigma_\epsilon^2 & \text{ for } s=1,\\ 0 & \text{ for } s \geq 2. \end{cases} \end{equation}

Therefore the equivalent representation would be ARIMA(0,1,1) and we have to choose parameters $\theta, \sigma_\nu^2$ so that the autocovariances above match the autocovariances $(1+\theta^2)\sigma^2_\nu$, $\theta\sigma^2_\nu$ of the process

\begin{align} \Delta y_t &= \nu_t + \theta \nu_{t-1}\\ \nu_t &\sim N(0,\sigma_\nu^2). \end{align}

It is straightforward to see the relationship between signal-to-noise ratio $\xi$ and MA weight $\theta$:

$$ \frac{\theta}{1+\theta^2} = -\frac{1}{2+\xi} \iff \xi = - 2 - \frac{1}{\theta} - \theta $$

It is convenient to plot the last relation:

The plot shows clearly three things:

1. For $\theta$ to be positive, the signal-to-noise ratio of the state-space model must be negative, which contradicts the physical sense of the parameters. In this sense, there is no unit-root state-space representation for the ARIMA(0,1,1) models with positive MA weights.

2. For any magnitude of signal-to-noise ratio there will be mean reversion in the MA part of the equivalent ARIMA representation, or equivalently, $\theta < 0$ always.

3. As signal-to-noise ratio increases, the delayed effect of shock $\nu$ in the equivalent representation becomes smaller, i.e. the absolute magnitude of $\theta$ decreases.

Regarding the variance of the shock $\nu$ we have the following relation:

$$ \sigma_\nu^2 = -\frac{1}{\theta} \sigma_\epsilon^2 $$

which futher suggests that, holding the variance of the noise $\sigma_\epsilon^2$ constant,

4. A signal-to-noise-ratio increases, the variance of the shock $\nu$ in the equivalent ARIMA representation increases.