Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have a very basic question. Please let me know if this has been asked before, but in my defence I haven't seen it on Cross Validated.

I understand that if a process depends on previous values of itself then it is an AR process. If it depends on previous errors than it is an MA process.

My question is, when would one of either of these two situations occur? Does anyone have a solid example that illuminates the underlying issue regarding what it means for a process to be best modelled as MA vs AR?

In my search for some sort of intuitive or dare I say philosophical insights into the difference between these two models, I keep running into material where an author makes a statement along the line of the above, and then immediately moves to discussing diagnostics: acf, pacf, etc.

Although I have the gut feeling that there is no simple answer to my question, if anyone can either enlighten me or point me to a good resource, I'd be grateful.

share|improve this question
1  
It's not as simple a dichotomy as that; after all, an AR can be written as an infinite MA and an (invertible) MA can be written as an infinite AR, so if either is ever appropriate, arguably so is the other. –  Glen_b Jul 14 at 6:13
    
Glen_b, can you elaborate on this? I understand it's not a simple dichotomy...am I correct to assume (hope, even) that there is something worth uncovering here? I don't want to simply run acf / pacf and pretend I have a good grasp on this process. –  Matt O'Brien Jul 14 at 9:23

3 Answers 3

One important and useful result is the Wold representation theorem (sometimes called the Wold decomposition), which says that every covariance-stationary time series $Y_{t}$ can be written as the sum of two time series, one deterministic and one stochastic.

$Y_t=\mu_t+\sum_{j=0}^\infty b_j \varepsilon_{t-j}\,$, where $\mu_t$ is deterministic.

The second term is an infinite MA.

(It's also the case that an invertible MA can be written as an infinite AR process.)

The relationship between an AR and the corresponding infinite MA is discussed in Hyndman and Athana­sopou­los' Forecasting: principles and practice, here

share|improve this answer

So you have a univariate time series and you want model it/forecast it, right? You have chosen to use an ARIMA type model.

The parameters of the depend on what's best for your dataset. But how do you find out? A recent approach is "Automatic time series forecasting" by Hyndman & Khandakar (2008) (pdf).

The algorithm tries different versions of p, q, P and Q and chooses the one with the smallest AIC, AICc or BIC. It is implemented in the auto.arima() function of the forecast R package. The choice of information criterion depends on the which parameters you pass to the function.

For a linear model, choosing a model with the smallest AIC can equivalent to leave-one-out cross-validation.

You should also make sure that you have enough data, at least four years.

Some important checks:

  1. Does the model make sense? For example, if you have monthly retails sales, you will probably expect a seasonal model to be fit.
  2. How well does it forecast out of sample?
share|improve this answer

The most common way to determine whether an AR or MA is appropriate is by looking at the autocorrelation functions of the data. My answer in this post describes the strategy to do so. It is basically based on the following features of AR and MA processes: 1) the ACF of a stationary AR process of order p goes to zero at an exponential rate, while the PACF becomes zero after lag p. 2) For an MA process of order q the theoretical ACF and PACF exhibit the reverse behaviour (the ACF truncates after lag q and the PACF goes to zero relatively quickly).

For processes that contain both AR and MA parts, the autocorrelation functions may not be clear. One way to proceed is to fit first an AR or MA model of low order and look at the autocorrelation functions of the residuals to see if there is some AR or MA structure that should be added to the initial model.

share|improve this answer
    
This type of approach is exactly what prompted me to create this post. Why, after being posed with the option of MA or AR, do we immediately jump to diagnostics? I'm curious: what properties of data might we have, that would lead to either an MA or AR model being fit, as will be ultimately be revealed by the ACF and PACF when we run the diagnostics? –  Matt O'Brien Jul 14 at 8:47
    
ARIMA modelling is about modelling the correlation patterns observed in the data. That's why the ACF and the PACF play an important role in the analysis; they reveal the properties of the data (the autocorrelation structure). If the model that is proposed fits the data well, then we expect the residuals to be white noise. For example, if the true model is an ARMA(2,1) and we fit an AR(2), then we will find that the residuals have a MA(1) structure and will realize that we should add this term to the initial model. –  javlacalle Jul 14 at 11:08
    
Be aware that not all AR models and not all MA models share same characteristics, even if we compare models of the same order. Depending on the coefficients of the model, cycles of different periodicity are more relevant in the data. You can simulate some AR or MA models with different parameters and compare the ACF-PACF and the periodogram of the simulated data. –  javlacalle Jul 14 at 11:11

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.