Types of noise processes and the one assumed in arima() estimation in R

Question

Here is a time series class defining white noise incorrectly as an independent sequence of random variables.

source

Aside from the widespread mix-up of White noise and iid noise, a further complicating factor in understanding the noise process is that we almost never seem to be given all the information about it. In ARMA models it is most often assumed to be:

1. Gaussian White noise (uncorrelated and Normally distributed)
2. Gaussian iid noise (independent and Normally distributed)
3. White noise (uncorrelated and distribution unspecified)
4. iid noise (independent and distribution unspecified).

Some estimation methods, like Maximum Likelihood or Kalman filter, require the specification of a distribution for the noise, while other, such as OLS for a pure AR model, do not require an assumption about the distribution of the noise process.

Let's take the arima() function in R. The default estimation method in arima() is described as follows: "The default (unless there are missing values) is to use conditional-sum-of-squares to find starting values, then maximum likelihood." Since it is Maximum Likelihood estimation, it is reasonable to assume that the noise is taken to be Gaussian, but there is however no mention of whether the noise is assumed to be Gaussian White noise or Gaussian iid noise. Can someone point me to where this distinction is made or explain why the type of noise assumed must be self-evident? While uncorrelatedness and joint normality imply independence, I don't believe we assume joint normality, just marginal normality - so there should still be a distinction between Gaussian White noise and Gaussian iid noise. See this Wikipedia article for more on this distinction.

Or the same question with a different flavor: draw 2 observations from the particular AR(2) process below:

arima.sim(n=2,list(ar=c(1.2,-0.8)),sd=0.5)

What have I just implicitly assumed about the noise process? Normally distributed, I presume. But iid noise or just white noise?

Answer 1

Firstly, I am not aware of any situation where someone would be crazy enough to use a time-series model where the noise terms are marginally Gaussian but not jointly Gaussian. So while it is true that one can construct uncorrelated but not independent Gaussian terms, these are not used in time-series modelling. In time-series modelling, if we use Gaussian noise at all, we will always assume that the noise sequence is Gaussian (i.e., each finite vector of noise terms has a multivariate Gaussian distribution). Since this distribution is fully defined by its first two moments, uncorrelated noise is identical to IID noise in this case. Consequently, your (1) collapses down to (2) in practice. As to the remaining distinctions in your taxonomy, as in regression modelling (see related answer here), it is possible to obtain certain useful results for ARIMA models without specifying a noise distribution (i.e., just specifying the first two moments of the noise) but it is possible to get more results by specifying the full distribution.

As you correctly point out, often the specification of time-series models is a bit sparse, and various things are often not specified explicitly. (Time-series texts are actually quite notorious for this.) By convention, unless there are contextual cues or direct specification to the contrary, you should assume that the model is jointly Gaussian if it uses any methods that require specification of the full distribution. So any time you see maximum likelihood estimation (MLE) used in an ARIMA model, without explicit specification of a noise distribution, you can reasonably take this to mean that the assumed noise sequence is Gaussian (i.e., all finite vectors of noise elements have a multivariate Gaussian distribution).