20
$\begingroup$

I understand that if a process depends on previous values of itself, then it is an AR process. If it depends on previous errors, then it is an MA process.

When would one of either of these two situations occur? Does anyone have a solid example that illuminates the underlying issue regarding what it means for a process to be best modeled as MA vs AR?

$\endgroup$
  • 3
    $\begingroup$ It's not as simple a dichotomy as that; after all, an AR can be written as an infinite MA and an (invertible) MA can be written as an infinite AR, so if either is ever appropriate, arguably so is the other. $\endgroup$ – Glen_b Jul 14 '14 at 6:13
  • 1
    $\begingroup$ Glen_b, can you elaborate on this? I understand it's not a simple dichotomy...am I correct to assume (hope, even) that there is something worth uncovering here? I don't want to simply run acf / pacf and pretend I have a good grasp on this process. $\endgroup$ – Matt O'Brien Jul 14 '14 at 9:23
  • $\begingroup$ Very much related: Real-life examples of moving average processes $\endgroup$ – Stephan Kolassa Mar 15 '18 at 7:54
7
$\begingroup$

One important and useful result is the Wold representation theorem (sometimes called the Wold decomposition), which says that every covariance-stationary time series $Y_{t}$ can be written as the sum of two time series, one deterministic and one stochastic.

$Y_t=\mu_t+\sum_{j=0}^\infty b_j \varepsilon_{t-j}\,$, where $\mu_t$ is deterministic.

The second term is an infinite MA.

(It's also the case that an invertible MA can be written as an infinite AR process.)

This suggests that if the series is covariance-stationary, and if we assume you can identify the deterministic part, then you can always write the stochastic part as an MA process. Similarly if the MA satisfies the invertibility condition you can always write it as an AR process.

If you have the process written in one form you can often convert it to the other form.

So in one sense at least, for covariance stationary series, often either AR or MA will be appropriate.

Of course, in practice we would rather not have very large models. If you have a finite AR or MA, both the ACF and PACF eventually decay away geometrically (there's a geometric function that the absolute value of either function will sit below), which will tend to mean that a good approximation of either an AR or an MA in the other form may often be reasonably short.

So under the covariance stationary condition and assuming we can identify the deterministic and stochastic components, often both AR and MA may be appropriate.

Box and Jenkins methodology looks for a parsimonious model -- an AR, MA or ARMA model with few parameters. Typically the ACF and PACF are used to try to identify a model, by transforming to stationarity (perhaps by differencing), identifying a model from the appearance of the ACF and PACF (sometimes people use other tools), fitting the model and then examining the structure of the residuals (typically via the ACF and PACF on the residuals) until the residual series appears reasonably consistent with white noise. Often there will be multiple models that can provide a reasonable approximation to a series. (In practice other criteria are often considered.)

There are some grounds for criticism of this approach. For one example, the p-values that result from such an iterative process don't generally take account of the way the model was arrived at (by looking at the data); this issue might be at least partly avoided by sample splitting, for example. A second example criticism is the difficulty of actually obtaining a stationary series - while one may in many cases transform to obtain a series that seems reasonably consistent with stationarity, it's not usually going to be the case that it really is (similar issues are a common problem with statistical models, though perhaps it may sometimes be more of an issue here).

[The relationship between an AR and the corresponding infinite MA is discussed in Hyndman and Athana­sopou­los' Forecasting: principles and practice, here]

$\endgroup$
  • $\begingroup$ -1 because, while it's sort of interesting, it doesn't really answer the spirit of the question. $\endgroup$ – Jake Westfall Jul 14 '18 at 19:04
  • $\begingroup$ Hi Jake -- thanks for adding the comment about what you think is wrong with the answer. This is much more helpful than a downvote alone would be. I agree there's something lacking here - at the very least it should be made clear why I thought that's relevant enough to post as an answer. $\endgroup$ – Glen_b Jul 14 '18 at 20:04
  • 1
    $\begingroup$ @jake I have made some edits which I hope make the connection to the question clearer. Thanks again for your help $\endgroup$ – Glen_b Jul 14 '18 at 20:36
  • $\begingroup$ Thanks for the edits. I removed my downvote. My one-sentence condensation of your revised answer would be something like: "This is a difficult question to answer in general because, in a lot of cases, either an AR or MA model could fit the data just about as well as the other." Which is a legitimate, if disappointing, response to the question. $\endgroup$ – Jake Westfall Jul 14 '18 at 22:47
  • $\begingroup$ @Jake don't feel the need to remove the downvote if you still have reservations; I appreciate the chance to improve the answer either way. $\endgroup$ – Glen_b Jul 14 '18 at 22:57
5
$\begingroup$

I can provide what I think is a compelling answer to the first part of the question ("whence MA?") but am presently pondering an equally compelling answer to the second part of the question ("whence AR?").

Consider a series consisting of the closing price (adjusted for splits and dividends) of a stock on consecutive days. Each day's closing price is derived from a trend (e.g., linear in time) plus the weighted effects of the daily shocks from prior days. Presumably, the effect of the shock at day t-1 will have a stronger influence on the price at day t than will the shock at day t-2, etc. Thus, logically, the stock's closing price at day t will reflect the trend value on day t plus a constant (less than 1) times the weighted sum of the shocks up through day t-1 (i.e., the error term at day t-1)(MA1), possibly plus a constant (less than 1) times the weighted sum of the shocks up through day t-2 (i.e., the error term at day t-2)(MA2), ..., plus the novel shock at day t (white noise). This kind of model seems appropriate for modelling series like the stock market, where the error term at day t represents the weighted sum of prior and current shocks, and defines an MA process. I am working through an equally compelling rationale for an exclusively-AR process.

$\endgroup$
  • 3
    $\begingroup$ +1. This is the only answer so far that even attempts to answer the original question. $\endgroup$ – Stephan Kolassa Mar 15 '18 at 7:54
-1
$\begingroup$

So you have a univariate time series and you want model it/forecast it, right? You have chosen to use an ARIMA type model.

The parameters of the depend on what's best for your dataset. But how do you find out? A recent approach is "Automatic time series forecasting" by Hyndman & Khandakar (2008) (pdf).

The algorithm tries different versions of p, q, P and Q and chooses the one with the smallest AIC, AICc or BIC. It is implemented in the auto.arima() function of the forecast R package. The choice of information criterion depends on the which parameters you pass to the function.

For a linear model, choosing a model with the smallest AIC can equivalent to leave-one-out cross-validation.

You should also make sure that you have enough data, at least four years.

Some important checks:

  1. Does the model make sense? For example, if you have monthly retails sales, you will probably expect a seasonal model to be fit.
  2. How well does it forecast out of sample?

Explicit answer to Firebug's comment below: When your data supports it.

$\endgroup$
  • 3
    $\begingroup$ This answer doesn't answer the question at all: "My question is, when would one of either of these two situations occur? " $\endgroup$ – Firebug Nov 15 '17 at 12:43
  • 1
    $\begingroup$ "Explicit answer to Firebug's comment below: When your data supports it." I agree with @Firebug - this is not an answer to the question and definitely not a solid example for differentiating between the two... $\endgroup$ – Thomas Mar 26 '18 at 13:00
  • $\begingroup$ xkcd.com/386 $\endgroup$ – power Mar 29 '18 at 5:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.