# Parameter space restriction in random walk + noise model

Suppose we have a random walk + noise model so

\begin{align} y_t & = \mu_{t-1} + \epsilon_t\\ \mu_t & = \mu_{t-1} + \eta_t \end{align}

Then, it's straightforward to show that

$$\triangle y_t = \eta_{t-1} + \epsilon_t - \epsilon_{t-1}$$

and also that, the autocorrelation at lag 1 of $$\Delta y_t$$, $$\gamma_1 = - \frac{\sigma^2_{\epsilon}}{\sigma^2_\eta + 2 \sigma^2_{e}}$$

So, the lag 1 autocorrelation has to be between $$-0.5$$ and $$0$$ by inspection.

But, consider the "equivalent" ARIMA(0,1,1) model:

$$\triangle y_t = \theta \nu_{t-1} + \nu_t$$

Since, $$\theta$$ has to be between -1 and + 1 in the ARIMA(0,1,1) and the first order autocorrelation is $$\frac{\theta}{1+\theta^2}$$, the first order autocorrelation is between -0.5 and 0.5.

So, in the random walk + noise formulation, the $$\theta$$ is restricted to be between $$-1$$ and $$0$$ and, in the ARIMA(0,1,1) formulation, $$\theta$$ is between $$-1$$ and $$1$$.

So, my only point is that there is a restriction for the random walk + noise model. But, if we put $$\phi$$ as a coefficient of $$\mu_t$$ in the random walk + noise model so that it's no longer a random walk + noise model, then the restriction goes away.

So, by enforcing the unit root for $$\mu_{t}$$, one ends up with a restriction in the "equivalence" between the random walk + noise model and the ARIMA(0,1,1) model. In this sense, the two formulations are not truly equivalent because there are regions of the ARIMA(0,1,1) parameter space that cannot be reached by the supposed equivalent random walk + noise model.

Does this make intuitive sense to anyone that the restriction goes away by the introduction of the new parameter $$\phi$$. Just curious? Thanks.

• @PsychhomeTstats: Thanks for the edits. Nov 24, 2019 at 14:21
• How is the kalman-filter tag relevant here? Nov 24, 2019 at 17:07
• @Richard Hardy: The random walk + noise model is a state space model so it's estimated using the kalman filter. dynamic linear model, state space representation and kalman filter are all kind of similar but I guess it depends on which field one is in. Terminology with respect to the KF can definitely be confusing because it spans so many different fields. Nov 24, 2019 at 20:28
• Also, since Richard might be referring to the following, I'll mention it. I wrote the repesentation so that the time is aligned in the observation equation and the system equation. That is not necessarily the usual way of writing the ss representation so it may be confusing. Just note that it is equivalent to the usual way of writing the ss where the times in the two equations are not aligned. The updating equations of course end up being different but they are not relevant to my question so it's okay to write it that way with the time aligned. Nov 24, 2019 at 20:32
• Thanks for the clarification. I am aware that they are all related, I just wondered if there was anything specific to the Kalman filter in the question (treating Kalman filter as estimation algorithm vs. state space as a model representation.) Nov 26, 2019 at 13:09

Every unit-root state-space model has an equivalent ARIMA(0,1,1) representation, but not every ARIMA(0,1,1) model has an equivalent state-space model representation.

The following holds true for the representations of unit-root state-space models:

1. Whatever the magnitude of the signal-to-noise ratio $$\frac{\sigma_\eta^2}{\sigma_\epsilon^2}$$ in the state-space model, MA weight $$\theta$$ of the MA term in the equivalent ARIMA representation lies in the interval $$(-1,0)$$.

Moreover, as the signal-to-noise ratio $$\frac{\sigma_\eta^2}{\sigma_\epsilon^2}$$ increases

1. The magnitude ($$|\theta|$$) of the delayed effect of the shocks in the equivalent representation decreases, and
2. The variance of the shock in the equivalent representation $$\sigma_\nu^2$$ must increase.

Discussion:

When talking about state-space models it is often convenient to introduce the signal-to-noise ratio, $$\xi := \frac{\sigma^2_\eta}{\sigma^2_\epsilon}$$. Let us try to see how this metric of the model affects the equivalent ARIMA representation.

ARIMA representation can be found by the method of undetermined coefficients. We know that $$\Delta y_t$$ has autocovariances

$$$$Cov(\Delta y_t, \Delta y_{t-s}) = \begin{cases} (\xi + 2)\sigma_\epsilon^2 & \text{ for } s=0,\\ -\sigma_\epsilon^2 & \text{ for } s=1,\\ 0 & \text{ for } s \geq 2. \end{cases}$$$$

Therefore the equivalent representation would be ARIMA(0,1,1) and we have to choose parameters $$\theta, \sigma_\nu^2$$ so that the autocovariances above match the autocovariances $$(1+\theta^2)\sigma^2_\nu$$, $$\theta\sigma^2_\nu$$ of the process

\begin{align} \Delta y_t &= \nu_t + \theta \nu_{t-1}\\ \nu_t &\sim N(0,\sigma_\nu^2). \end{align}

It is straightforward to see the relationship between signal-to-noise ratio $$\xi$$ and MA weight $$\theta$$:

$$\frac{\theta}{1+\theta^2} = -\frac{1}{2+\xi} \iff \xi = - 2 - \frac{1}{\theta} - \theta$$

It is convenient to plot the last relation:

The plot shows clearly three things:

1. For $$\theta$$ to be positive, the signal-to-noise ratio of the state-space model must be negative, which contradicts the physical sense of the parameters. In this sense, there is no unit-root state-space representation for the ARIMA(0,1,1) models with positive MA weights.

2. For any magnitude of signal-to-noise ratio there will be mean reversion in the MA part of the equivalent ARIMA representation, or equivalently, $$\theta < 0$$ always.

3. As signal-to-noise ratio increases, the delayed effect of shock $$\nu$$ in the equivalent representation becomes smaller, i.e. the absolute magnitude of $$\theta$$ decreases.

Regarding the variance of the shock $$\nu$$ we have the following relation:

$$\sigma_\nu^2 = -\frac{1}{\theta} \sigma_\epsilon^2$$

which futher suggests that, holding the variance of the noise $$\sigma_\epsilon^2$$ constant,

1. A signal-to-noise-ratio increases, the variance of the shock $$\nu$$ in the equivalent ARIMA representation increases.
• Thanks a lot. I'll go over it carefully this week. It's much appreciated. Dec 2, 2019 at 15:10