Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

This morning I woke up wondering (this could be due to the fact that last night I didn't get much sleep): since cross-validation seems to be the cornerstone of proper time-series forecasting, what are the models I should "normally" cross-validate against?

I came up with a few (easy) ones, but I soon realized they were all but special cases of ARIMA models. So I'm now wondering, and this is the actual question, what forecasting models does the Box-Jenknins approach already incorporate?

Let me put it this way:

  1. Mean = ARIMA(0,0,0) with constant
  2. Naive = ARIMA(0,1,0)
  3. Drift = ARIMA(0,1,0) with constant
  4. Simple Exponential Smoothing = ARIMA(0,1,1)
  5. Holt's Exponential Smoothing = ARIMA(0,2,2)
  6. Damped Holt's = ARIMA(0,1,2)
  7. Additive Holt-Winters: SARIMA(0,1,m+1)(0,1,0)m

What else can be added to the previous list? Is there a way to do moving average or least squares regression "the ARIMA way"? Also how do other simple models (say ARIMA(0,0,1), ARIMA(1,0,0), ARIMA(1,1,1), ARIMA(1,0,1), etc.) translate?

Please note that, at least for starters, I'm not interested in what ARIMA models cannot do. Right now I only want to focus on what they can do.

I know that understanding what each "building block" in an ARIMA model does should answer all of the above questions, but for some reason I have difficulties figuring that out. So I dediced to try a "reverse engineering" kind of approach.

Any input is appreciated, I hope I stated my question clearly enough.

share|improve this question

3 Answers 3

up vote 3 down vote accepted

:Bruder the Box-Jenknins approach incorporates all well-known forecasting models except multiplicative models like the Holt-Winston Multiplicative Seasonal Model where the expected value is based upon a multiplicand . The multiplicative seasonal model can be used for to model time series where one has the following ( in my opinion a very unusual) case . If the amplitude of the seasonal component/pattern is proportional to the average level of the series, the series can be referred to as having multiplicative seasonality. Even in the case of multiplicative models, one can often represent these as ARIMA models http://support.sas.com/documentation/cdl/en/etsug/60372/HTML/default/viewer.htm#etsug_tffordet_sect014.htm thus completing the "umbrella". Furtermore since a Transfer Function is a Generalized Least Squares Model it can reduce to a standard regression model by omitting the ARIMA component and assuming a set of weights needed to homogenize the error structure.

share|improve this answer
    
I lost you here: "it can reduce to a standard regression model by omitting the ARIMA component and assuming a set of weights needed to homogenize the error structure". Otherwise thank you for your answer and link. Also, can't multiplicative models be mimicked via a log trasformation? I read somewhere (bottom of the page) that logging can help in this regard. –  Bruder Feb 29 '12 at 13:45
    
:Bruder A Transfer Function (multivariate Box-Jenkins) can have a PDL(polynomial distributed lag) structure on user-specified input series with an ARIMA component reflecting user-omitted stochastic input series.If you eliminate the ARIMA component you have a lagged regression structure. Often one needs to render the error variance homoegenous via either power transforms (e.g. logs ) or weighted least squares where weights are applied (GLS).These are easily handled via Box-Jenkins.Note that a Log Transform does not ALWAYS deal with data that is fundamentally a multiplicative model. –  IrishStat Feb 29 '12 at 14:15
    
Isn't ARIMA(1,0,0) a regression model where Y = a + b Y_t-1? –  zbicyclist Feb 29 '12 at 14:24
1  
:zbicylist Correct, since this is a special case of a Transfer Function where there are no user specified inputs and the form of the ARIMA model is (1,0,0) and the model assumes that there are no deterministic variables to be empirically identified ( such as Pulses, level Shifts , Seasonal Pulses and/or Local Time Trends via Intervention Detection . –  IrishStat Feb 29 '12 at 15:35
    
Ok, so to fit a simple least-squares line through the points in my scatterplot all I need is an ARIMA(1,0,0) model? If so I'll add it to the list above. And what about moving average? Is it simply an ARIMA(0,0,1)? If so, how do I choose the width of the moving average window? And what is the difference between an ARIMA(0,0,1) and an ARIMA(0,0,1) with constant. Again, I'm sorry if the answer seems obvious to everyone but me :) –  Bruder Feb 29 '12 at 16:00

You can add

Drift: ARIMA(0,1,0) with constant.

Damped Holt's: ARIMA(0,1,2)

Additive Holt-Winters: SARIMA(0,1,$m+1$)(0,1,0)$_m$.

However, HW uses only three parameters and that (rather strange) ARIMA model has $m+1$ parameters. So there are a lot of parameter constraints.

The ETS (exponential smoothing) and ARIMA classes of models overlap, but neither is contained within the other. There are a lot of non-linear ETS models that have no ARIMA equivalent, and a lot of ARIMA models that have no ETS equivalent. For example, all ETS models are non-stationary.

share|improve this answer
  • The exponentially weighted moving average (EWMA) is algebraically equivalent to an ARIMA(0,1,1) model.

To put it another way, the EWMA is a particular model within the class of ARIMA models. In fact, there are various types of EWMA models and these happen to be included in the class of ARIMA(0,d,q) models - see Cogger (1974):

The Optimality of General-Order Exponential Smoothing by K. O. Cogger. Operations Research. Vol. 22, No. 4 (Jul. - Aug., 1974), pp. 858-867.

The abstract for the paper is as follows:

This paper derives the class of nonstationary time-series representations for which exponential smoothing of arbitrary order minimizes mean-square forecast error. It points out that these representations are included in the class of integrated moving averages developed by Box and Jenkins, permitting various procedures to be applied to estimating the smoothing constant and determining the appropriate order of smoothing. These results further permit the principle of parsimony in parameterization to be applied to any choice between exponential smoothing and alternative forecasting procedures.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.