# Is ARMA fit well-defined?

My question is whether unique time series has a unique set of ARMA parameters that fit it best, once order of AR and MA have been chosen.

For simplicity, I will ask only about ARMA(1,1) process. Lets say I have time series $$\{x_t\}_{t=0\dots N-1}$$. I can define the lag operator as a matrix:

$$L=\left(\begin{array}\\ 0 & 1 & 0 & 0 & \dots & 0 & 0 \\ 0 & 0 & 1 & 0 & \dots & 0 & 0\\ \dots \\ 0 & 0 & 0 & 0 & \dots & 1 & 0 \\ 0 & 0 & 0 & 0 & \dots & 0 & 1 \\ 0 & 0 & 0 & 0 & \dots & 0 & 0 \\ \end{array}\right)$$

where $$L^k=0$$ for all $$k\ge N$$. Then

$$L\left(\begin{array}\\x_{N-1} \\ x_{N-2}\\ \dots\\ x_1 \\ x_0 \end{array}\right)= \left(\begin{array}\\ x_{N-2} \\ x_{N-3} \\ \dots\\ x_0 \\ 0 \end{array}\right).$$

With this, I can describe ARMA process as one that fits

$$\left(1-\alpha\cdot L\right)\left(\begin{array}\\x_{N-1} \\ x_{N-2}\\ \dots\\ x_1 \\ x_0 \end{array}\right)=\left(1-\theta\cdot L\right)\left(\begin{array}\\n_{N-1} \\ n_{N-2}\\ \dots\\ n_1 \\ n_0 \end{array}\right)$$

with $$-1<\theta,\alpha<1$$ and $$n_{\dots}\sim\mathcal{N}\left(0,\,1\right)$$ being iid normal variables.

Under these conditions $$\left(1-\alpha\cdot L\right)$$ and $$\left(1-\theta\cdot L\right)$$ are invertible so I can also write:

$$\left(1-\theta\cdot L\right)^{-1}\cdot\left(1-\alpha\cdot L\right)\left(\begin{array}\\x_{N-1} \\ x_{N-2}\\ \dots\\ x_1 \\ x_0 \end{array}\right)=\left(\begin{array}\\n_{N-1} \\ n_{N-2}\\ \dots\\ n_1 \\ n_0 \end{array}\right).$$

It would therefore seem that if time series $$x_{t}$$ is normally distributed already, i.e. $$x_t\sim \mathcal{N}\left(0,\,1\right)$$ irrespective of time, then any combination of $$\theta=\alpha$$ parameters will fit the solution. If I tried to fit $$\alpha,\theta$$ using maximum likelihood in such case, it would seem that optimizer would have an infinite number of equally valid solutions (though some may behave better numerically).

I am guessing this problem will only become more obvious as the order of the time-series grows. How is this problem normally solved? Is there expectation for ARMA constants to be well-defined given the time-series?

• if time series $x_t$ is normally distributed already, i.e. it is simply noise... How do you define "simply noise" and why is the type of distribution (normal vs. nonnormal) relevant? Commented Aug 11, 2023 at 8:20
• @RichardHardy I meant if $x_t\sim N\left(0,\,1\right)$ already, irrespective of time.
– Cryo
Commented Aug 11, 2023 at 9:39
• Are you asking whether ARMA(p,q) always leads to a unique process? The answer is NO. Different specifications ARMA(p,q) and ARMA(p1,q1) may lead to the same process. Commented Aug 11, 2023 at 13:24
• for AR(1) process to be stationary $\alpha\ne 1$ but to be causal $\alpha<1$. the "direction" is not the concept from time series analysis. I know it's a weird notion that the errors come from future. think of it as a math trick. generally you write ARMA as $\phi(B)x_t=\theta(B)e_t$, so you can represent it either as AR or as MA, e.g. $x_t=\phi^{-1}(B)\theta(B)e_t$ Commented Aug 13, 2023 at 4:32
• so you get into weird situation sometimes like ARMA(2,1) process: $x_t=1/4x_{t-2}+e_t+1/2e_{t-1}$. in this case the polynomials have a common root because $1-1/4=(1-1/2)(1+1/2)$, so the process is really AR(1): $x_t=1/2x_{t-1}+e_t$, i.e. we have different set of coefficients that produce the same process Commented Aug 13, 2023 at 4:45