There are many ARMA(p,q) processes, with different parameter values, associated with the same auto-correlation function. If this processes are also Gaussian, this implies they follow same exact distribution.

So, for a set of data points, there are usually multiple parameter values associated with the same (maximum) likelihood. Only one of this processes is also invertible, we choose that particular estimate restricting the parameter space to that of invertible processes (making the model identifiable).

If the processes are not Gaussian, it is still true that there are multiple processes with different parameters all having the same auto-correlation function, but, I don't think this implies they follow the same distribution as before.

So, for a set of data points, I wouldn't expect to have multiple points in the parameter space with the same maximum likelihood. Is this correct?

Restricting the parameter space was useful with Gaussian processes since it was a way of choosing the invertible representation of the process among all those having the same maximum likelihood. Now, with non-Gaussian processes, the possibly unique maximum likelihood estimate could be an invertible process or not. If it isn't, we can't actually choose an invertible representation with the same maximum likelihood.

In this case, is it a problem to have a process that isn't invertible?


1 Answer 1


Yes, for non-Gaussian white noise, the distribution of the data may be different for an invertible model and its non-invertible counterpart(s). For example, consider the invertible MA(1) model $$ y_t = w_t + \frac12 w_{t-1} \tag{1} $$ where the white noise process $w_t \sim U(-1,1)$ and the non-invertible model $$ y_t = w_t + 2 w_{t-1} \tag{1} $$ where $w_t \sim U(-1/2,1/2)$. Although these processes have the same autocovariance functions the joint distribution of for instance $y_1,y_2$ are different as demonstrated by the following simulation.

n <- 1e+4
# Invertible version
s <- .5
w0 <- runif(n,-s,s)
w1 <- runif(n,-s,s)
w2 <- runif(n,-s,s)
y1 <- w1 + .5*w0
y2 <- w2 + .5*w1
# Non-invertible counterpart
s <- .25
w0 <- runif(n,-s,s)
w1 <- runif(n,-s,s)
w2 <- runif(n,-s,s)
y1 <- w1 + 2*w0
y2 <- w2 + 2*w1

Given enough data, you should thus in principle be able to distinguish between models such as (1) and (2) and relying on (2) e.g. for prediction would of course be perfectly fine. But the exact likelihood seems rather hard to compute. Also, depending on the exact distribution of the white noise, the difference between the resulting joint distributions of the data may be much less obvious than in this toy example. But, yes, this sort of thing is something people have looked into.

  • $\begingroup$ So is it a problem to have a process that isn't invertible? $\endgroup$ Dec 22, 2021 at 9:22
  • $\begingroup$ @RichardHardy No, such non-Guassian, non-invertible models seems to be in wide use if you look into the literature. $\endgroup$ Dec 22, 2021 at 9:45
  • $\begingroup$ Thank you, that was helpful! $\endgroup$ Dec 22, 2021 at 10:23
  • $\begingroup$ @JarleTufto thank you! this example shows really well what I was kind of thinking about when writing the question. So, would you say that invertibility is necessary only for Gaussian processes, with the main goal of assuring identifiability? $\endgroup$ Dec 24, 2021 at 4:50
  • $\begingroup$ @JarleTufto Would you also kindly take a look at this related question I wrote later? (stats.stackexchange.com/q/557948/318559) the book by Box and Jenkins seems to make a big deal about invertibility and I don’t get why... $\endgroup$ Dec 24, 2021 at 4:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.