Proper ways to perform time series and ARIMA

Question

Note that I do most of my analysis using R and Excel.

Let's take this data set for example. I modified it as the data itself is proprietary: the years are also different:

1967    2,033,407
1968    2,162,275
1969    2,159,640
1970    2,312,352
1971    2,554,449
1972    2,548,425
1973    2,101,225
1974    1,951,944
1975    2,106,250
1976    1,687,625
1977    1,636,496
1978    1,494,525
1979    1,606,825
1980    1,460,937
1981    1,310,494
1982    1,319,750
1983    1,263,643
1984    1,171,656
1985    1,194,950

What I usually do:

1. A linear regression
2. Some form of polynomial trending
3. Moving average and double moving average
4. Basic ARIMA using p = 1, q = 0.
5. I calculate the errors for all these as well
6. I average all the forecasts out and the error to have my final result.

Note that I'm an engineer that wants to get into statistics and the ability to properly validate and calibrate my models.

Question

What is the correct way to forecast this to 5, 10, or even 15 future years?

In a way I'm looking to move beyond the plugging data into a model and believe the data. Yes, I'm aware I can look at the errors. I mainly use RMSE or MAE. But I still am not confident when it comes to just predicting data the right way.

Note

this is also related to this question I posted here before.

Answer 1

The answer here is the same as the previous one: cross-validate! Don't forget exponential smoothing models as well.

In general, the ets and auto.arima functions in the forecast package are pretty good across a wide range of data. The theta method did well in the M3 competition.

It might be a good idea to read some of the papers associated with the M3 competition, which will help you conceptualize how good forecasting models are evaluated.

Answer 2

Your data suggests a model of the form

[(1-B**1)]Y(T) = + [X1(T)][(1-B**1)][(-  .17233E+06)]   :PULSE               7
                 + [X2(T)][(1-B**1)][(+  .32470E+06)]   :PULSE               9
                 + [(1-  .488B** 1)]**-1  [A(T)]

ARIMA (1,1,0) with 2 anomalous data points at 7 and 9

yielding an rsq of

R Square  =  .916201