0
$\begingroup$

My question is whether unique time series has a unique set of ARMA parameters that fit it best, once order of AR and MA have been chosen.

For simplicity, I will ask only about ARMA(1,1) process. Lets say I have time series $\{x_t\}_{t=0\dots N-1}$. I can define the lag operator as a matrix:

$$ L=\left(\begin{array}\\ 0 & 1 & 0 & 0 & \dots & 0 & 0 \\ 0 & 0 & 1 & 0 & \dots & 0 & 0\\ \dots \\ 0 & 0 & 0 & 0 & \dots & 1 & 0 \\ 0 & 0 & 0 & 0 & \dots & 0 & 1 \\ 0 & 0 & 0 & 0 & \dots & 0 & 0 \\ \end{array}\right) $$

where $L^k=0$ for all $k\ge N$. Then

$$ L\left(\begin{array}\\x_{N-1} \\ x_{N-2}\\ \dots\\ x_1 \\ x_0 \end{array}\right)= \left(\begin{array}\\ x_{N-2} \\ x_{N-3} \\ \dots\\ x_0 \\ 0 \end{array}\right). $$

With this, I can describe ARMA process as one that fits

$$ \left(1-\alpha\cdot L\right)\left(\begin{array}\\x_{N-1} \\ x_{N-2}\\ \dots\\ x_1 \\ x_0 \end{array}\right)=\left(1-\theta\cdot L\right)\left(\begin{array}\\n_{N-1} \\ n_{N-2}\\ \dots\\ n_1 \\ n_0 \end{array}\right) $$

with $-1<\theta,\alpha<1$ and $n_{\dots}\sim\mathcal{N}\left(0,\,1\right)$ being iid normal variables.

Under these conditions $\left(1-\alpha\cdot L\right)$ and $\left(1-\theta\cdot L\right)$ are invertible so I can also write:

$$ \left(1-\theta\cdot L\right)^{-1}\cdot\left(1-\alpha\cdot L\right)\left(\begin{array}\\x_{N-1} \\ x_{N-2}\\ \dots\\ x_1 \\ x_0 \end{array}\right)=\left(\begin{array}\\n_{N-1} \\ n_{N-2}\\ \dots\\ n_1 \\ n_0 \end{array}\right). $$

It would therefore seem that if time series $x_{t}$ is normally distributed already, i.e. $x_t\sim \mathcal{N}\left(0,\,1\right)$ irrespective of time, then any combination of $\theta=\alpha$ parameters will fit the solution. If I tried to fit $\alpha,\theta$ using maximum likelihood in such case, it would seem that optimizer would have an infinite number of equally valid solutions (though some may behave better numerically).

I am guessing this problem will only become more obvious as the order of the time-series grows. How is this problem normally solved? Is there expectation for ARMA constants to be well-defined given the time-series?

Improve this question
$\endgroup$
17
  • $\begingroup$ if time series $x_t$ is normally distributed already, i.e. it is simply noise... How do you define "simply noise" and why is the type of distribution (normal vs. nonnormal) relevant? $\endgroup$
    – Richard Hardy
    Commented Aug 11, 2023 at 8:20
  • $\begingroup$ @RichardHardy I meant if $x_t\sim N\left(0,\,1\right)$ already, irrespective of time. $\endgroup$
    – Cryo
    Commented Aug 11, 2023 at 9:39
  • 1
    $\begingroup$ Are you asking whether ARMA(p,q) always leads to a unique process? The answer is NO. Different specifications ARMA(p,q) and ARMA(p1,q1) may lead to the same process. $\endgroup$
    – Aksakal
    Commented Aug 11, 2023 at 13:24
  • 2
    $\begingroup$ for AR(1) process to be stationary $\alpha\ne 1$ but to be causal $\alpha<1$. the "direction" is not the concept from time series analysis. I know it's a weird notion that the errors come from future. think of it as a math trick. generally you write ARMA as $\phi(B)x_t=\theta(B)e_t$, so you can represent it either as AR or as MA, e.g. $x_t=\phi^{-1}(B)\theta(B)e_t$ $\endgroup$
    – Aksakal
    Commented Aug 13, 2023 at 4:32
  • 1
    $\begingroup$ so you get into weird situation sometimes like ARMA(2,1) process: $x_t=1/4x_{t-2}+e_t+1/2e_{t-1}$. in this case the polynomials have a common root because $1-1/4=(1-1/2)(1+1/2)$, so the process is really AR(1): $x_t=1/2x_{t-1}+e_t$, i.e. we have different set of coefficients that produce the same process $\endgroup$
    – Aksakal
    Commented Aug 13, 2023 at 4:45

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.