I am quite new to time series analysis and I am delving for the first time into stationary processes. I don't seem to understand the concepts of non-stationarity and the presence of roots within the unit circle. Particularly, these are my questions:

  1. Where does the characteristic equation come from (i.e. what is its meaning) and why is it important? (I know the way it is built, I just do not understand why it is built that way)

  2. Why can we say that a process is stationary if and only if the roots of the characteristic equation are outside of the unit circle?

  1. Not restricted to time-series analysis, characteristic equations (CE) are used in many applications or problems, such as differential/difference equation solving, signal processing, control systems etc. And, it is directly related with commonly used transforms, e.g. Z, (Disc./Cont.) Fourier, Laplace transform etc. Using back-shift operator is a type of analysis allowing one to transform the time-series equations into another domain and deduce related properties. Most analyses are not easy to do in time-domain, which is why transforms exist.

    And, CE obtained by the backshift operator is a way of analyses. This is a matter of which transform you use. You could get the CE of $y_t+\alpha y_{t-1}+\beta y_{t-2}=\epsilon_t$ as $1+\alpha B+\beta B^2$ or if you use $\mathcal{Z}$ transform, that'd be $1+\alpha z^{-1}+\beta z^{-2}=0$, which requires your roots, i.e. $z$'s, should be inside the unit circle if you want to be stationary, instead of outside as in $B$, since roots obtained by $B$ and $z$ are reciprocals of each other.

  2. If we go back to what you're accustomed to, i.e. using the back-shift operator, $B$, we can try to find the roots' relation to stationarity. Consider the time series $y_t=2y_{t-1}+\epsilon_t$, we use the previous output, double it, and add it up with $\epsilon_t$ to obtain current output. The process is clearly exploding. The CE of this is $1-2B=0$, which yields $|B|=1/2<1$. But, if it were $y_t=0.5y_{t-1}+\epsilon_t$, the root would be $|B|=2>1$. This is for just an intuition why would you need $|B|>1$ for being stationary.

    Now consider the general case, $y_t=\alpha y_{t-1}+\epsilon_t$. We can obtain the following equation by back-substitution: i.e. $y_t=\alpha^2y_{t-2}+\alpha\epsilon_{t-1}+\epsilon_t$, and so... $$y_t=\sum_{i=0}^\infty{\alpha^i\epsilon_{t-i}}$$ mean and variance of $y_t$ would be $$E[y_t]=\sum_{i=0}^{\infty}{\alpha^iE[\epsilon_{t-i}]}=\mu_\epsilon\sum_{i=0}^\infty\alpha^i=\frac{\mu_\epsilon}{1-\alpha} \ \ \text{iff} \ \ |\alpha|<1 \rightarrow |B|>1$$ since CE is $1-B\alpha=0\rightarrow B=1/\alpha$. Similarly for the variance, we have $$\sigma_Y^2=var(y_t)=\sum_{i=0}^{\infty}\alpha^{2i}\sigma_{\epsilon}^2=\frac{\sigma_{\epsilon}^2}{1-\alpha^2} \ \ \text{iff} \ \ |\alpha^2|<1 \rightarrow |\alpha=1/B|<1$$

    The auto-correlation is $r_Y(k)=E[y_t y_{t-k}]:$ $$E[y_t y_{t-1}]=E[(\alpha y_{t-1}+\epsilon_t)y_{t-1}]=\alpha\sigma_Y^2+\mu_\epsilon\mu_Y$$ $$E[y_t y_{t-2}]=E[(\alpha y_{t-1}+\epsilon_t)y_{t-2}]=\alpha r_Y(1)+\mu_\epsilon\mu_Y=\alpha^2\sigma_Y^2+\mu_\epsilon\mu_Y(1+\alpha)$$ ... $$E[y_t y_{t-k}]=\alpha^k\sigma_Y^2+\mu_\epsilon\mu_Y(1+\alpha+...+\alpha^{k-1})$$ which is irrespective of $t$, and conditioned on $\sigma_Y^2$ and the means don't depend on $t$.

    This is for AR(1), but AR(k) can be reduced down to a series of AR(1)'s: $$(1-\alpha B)(1-\beta B)y_t=\epsilon_t \rightarrow x_t = (1-\beta B)y_t=x_t \ \ \ \& \ \ (1-\alpha B)x_t=\epsilon_t$$ and the analysis can be performed recursively. Here, we're actually referring to weak stationarity but if $\epsilon_t$ is Gaussian as usual (i.e. which is the noise process is assumed to be distributed with typically), weak stationarity corresponds to stationarity.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.