Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Note that I do most of my analysis using R and Excel.

Let's take this data set for example. I modified it as the data itself is proprietary: the years are also different:

1967    2,033,407
1968    2,162,275
1969    2,159,640
1970    2,312,352
1971    2,554,449
1972    2,548,425
1973    2,101,225
1974    1,951,944
1975    2,106,250
1976    1,687,625
1977    1,636,496
1978    1,494,525
1979    1,606,825
1980    1,460,937
1981    1,310,494
1982    1,319,750
1983    1,263,643
1984    1,171,656
1985    1,194,950

What I usually do:

  1. A linear regression
  2. Some form of polynomial trending
  3. Moving average and double moving average
  4. Basic ARIMA using p = 1, q = 0.
  5. I calculate the errors for all these as well
  6. I average all the forecasts out and the error to have my final result.

Note that I'm an engineer that wants to get into statistics and the ability to properly validate and calibrate my models.


What is the correct way to forecast this to 5, 10, or even 15 future years?

In a way I'm looking to move beyond the plugging data into a model and believe the data. Yes, I'm aware I can look at the errors. I mainly use RMSE or MAE. But I still am not confident when it comes to just predicting data the right way.


this is also related to this question I posted here before.

share|improve this question
up vote 7 down vote accepted

The answer here is the same as the previous one: cross-validate! Don't forget exponential smoothing models as well.

In general, the ets and auto.arima functions in the forecast package are pretty good across a wide range of data. The theta method did well in the M3 competition.

It might be a good idea to read some of the papers associated with the M3 competition, which will help you conceptualize how good forecasting models are evaluated.

share|improve this answer
Thanks for all the help on both answers Zach :) – dassouki Dec 9 '11 at 23:42
@dassouki: No problem! – Zach Dec 10 '11 at 15:35

Your data suggests a model of the form

[(1-B**1)]Y(T) = + [X1(T)][(1-B**1)][(-  .17233E+06)]   :PULSE               7
                 + [X2(T)][(1-B**1)][(+  .32470E+06)]   :PULSE               9
                 + [(1-  .488B** 1)]**-1  [A(T)]

ARIMA (1,1,0) with 2 anomalous data points at 7 and 9

yielding an rsq of

R Square  =  .916201               

enter image description here

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.