Definitions of coefficients from Arima {forecast}

Question

I'm trying to explain in detail step by step what my code does and I am stuck at explaining what the coefficients are in an Arima model and where they are from/what relevance they have.

Could someone please explain to me what ma1, sar1, sar2, ar1, ar2, ar3, sma1 are and if possible show with a formula where they may appear in the equation for an ARIMA process? (The equation is not high priority as long as they are explained well)

Here is an ARIMA model found using ?Arima just you can see what i mean by the coefficients

fit <- Arima(WWWusage,order=c(3,1,0))

Answer 1

In brief, the autoregressive (AR) terms represents the relationship between $y_t$ and $y_{t-1}$. A simple AR(1) model is:

$$ y_t=\phi_1 y_{t-1} + \epsilon_{t-1} $$

In words, if $y_{t-1}$ is large, subsequent $y$'s also tend to be large if $\phi>0$ (although, if $\phi$ is less than 1, then $y$ will tend to gradually collapse back down).

In an AR(p) process, this is extended to $p$ lagged $y$ terms.

Moving average (MA) terms arise from a model like this: $$ y_t = \theta_1 \epsilon_{t-1} + \epsilon_{t} $$

More generally, an MA(q) process is a moving average of the last $q$ error terms ....with weights equal to $\theta_1 \ldots \theta_q$.

A combination of AR and MA models is called an ARMA model.

Finally, having differences in the model (the middle term of the ARIMA model specification in R) means that instead of an ARMA model in $y$, the ARMA model describes $y_t-y_{t-1}$.

You also referred to sma1 and sar1 terms ... you can extend the ARIMA model even further to also cover seasonal time series, in which case sma1 and sar1 refer to the coefficients of the lagged errors and $y_t$'s at seasonal periods (ie 12 months ago for an annual model).

Rob Hyndman's excellent online textbook Forecasting Principles and Practice contains a chapter on ARIMA models that explains the meaning of the terms in far more detail than above. Other (offline) standard references include Applied Time Series Modelling and Forecasting (Harris) and Time Series Analysis (Hamilton).

Answer 2

The output tells you that the model chosen for your data is the following polynomial on lags of $y_t^*$:

$$ y_t^* = 1.15\, y_{t-1}^* -0.66\, y_{t-2}^* + 0.34\, y_{t-3}^* + \hbox{residuals} \,, \quad t=1,2,...,n \,, $$

where $y_t^* = y_t - y_{t-1}$, that is, the first differences of the original series $y_t$. This means that the chosen model considers the presence of a stochastic trend rather than a deterministic trend, e.g. linear trend.

As regards the coefficients, they are weights of past observations of the data (in this case of the first differences of the data). We may expect that these weights will decay or go to zero. The reason is that, in general, the current observation is more closely related to the most recent observations rather than to old observations. For example, the prices observed in the present year are more likely to be related to the prices observed in the previous year rather than to the prices observed 10 years ago.

The coefficients can be also interpreted in terms of the underlying cycles generated by the polynomial related to the ARMA model. You can get the roots of the polynomial and the frequency of the cycles generated by that polynomial. Frequencies close to zero will mean that long-term patterns (i.e., cycles that are repeated after many observations) explain part of the dynamics of the data and the series will look relatively smooth. More erratic series are explained by cycles of lower frequency that are repeated every fewer observations. In seasonal time series, it is common to see cycles related to seasonal frequencies that are repeated every 2, 6 or 12 moths. For an example that includes some of the calculus behind this interpretation you may see this post.