I am a beginner to time series modeling but I am trying to build an ARMA model to describe a set of 24 observations.


The ACF pattern seems to indicate that this is an AR process.

Graph of Autocorrelation Function

The PACF shows a significant corrleation at lag 2.

Graph of Partial Autocorrelation Function

Based on this, I fit a ARMA (2,0,0) model to the data

fitdata <- Arima(tsdata,c(2,0,0))

This resulted in the following model

Series: tsdata ARIMA(2,0,0) with non-zero mean

Coefficients: ar1 ar2 mean 0.1023 0.6691 -2.0834 s.e. 0.1489 0.1652 1.3511

sigma^2 estimated as 3.987: log likelihood=-49.69 AIC=107.39
AICc=109.49 BIC=112.1

I then tried to predict 6 time periods ahead but as you can see the predictions get consistently smaller over time and do not seem to capture the overall trend of the data. Can someone tell me what I'm doing wrong?


EDIT: I should have mentioned earlier that according to a Dickey Fuller test the current time series is stationary.

Augmented Dickey-Fuller Test

data: tsdata Dickey-Fuller = -4.0131, Lag order = 1, p-value = 0.0228 alternative hypothesis: stationary

enter image description here

  • 1
    $\begingroup$ If you have trend, is the assumption of stationarity reasonable? $\endgroup$ – Glen_b May 11 '19 at 2:51
  • $\begingroup$ @Glen_b I forgot to include my Dickey Fuller test earlier. But it seemed to indicate that the current model is stationary(added it to my post). I'm assuming that's because the test uses the model that includes a constant and linear trend. $\endgroup$ – Michael Howell May 11 '19 at 4:46
  • $\begingroup$ Your ARMA model has no trend in it, what good is a test that removes almost all the trend in the data for judging stationarity if you don't use a similar model?. $\endgroup$ – Glen_b May 11 '19 at 6:19
  • $\begingroup$ Would this be covered by the include.drift parameter? I had tried that before but it negatively shifted the prediction along the y axis. $\endgroup$ – Michael Howell May 11 '19 at 15:53
  • $\begingroup$ I was addressing an inconsistency in what you were saying; I don't actually believe you have something well modelled by AR+linear trend (though I do believe the mean is not constant). $\endgroup$ – Glen_b May 12 '19 at 2:13

Any stationary AR(2) process will converge towards the mean. You can see this by yourself by taking any initial values $x_1$ and $x_2$, then developing the future expectations by iteratively plugging in the fitted equation

$$ x_t=0.16x_{t-1}+0.7x_{t-2} $$

(where we set the intercept to $0$). For instance:

xx <- rnorm(2)
for ( ii in 3:10 ) xx <- c(xx,.16*xx[ii-1]+0.7*xx[ii-2])


More information here.

Incidentally, I would recommend against "rolling your own" ARIMA model. Much better to rely on trusted software, like forecast::auto.arima(). Which indeed finds a drift (i.e., trend) term here, along with a first order integration:



I find this forecast somewhat more convincing than the AR(2) one in the question.

| cite | improve this answer | |
  • $\begingroup$ Thank you this helps alot and I really appreciate the level of detail you provided. $\endgroup$ – Michael Howell May 11 '19 at 15:57
  • $\begingroup$ That model identified by auto.arima is essentially what I'd have guessed to be a reasonable ARIMA model from looking at the question (assuming ARIMA was all that was being considered). $\endgroup$ – Glen_b May 12 '19 at 2:16

ARIMA modeling procedures that ignore the possibility of anomalies often are quite deficient as suggested by @AdamO here Interrupted Time Series Analysis - ARIMAX for High Frequency Biological Data? ... "The correlogram should be calculated from residuals using a model that controls for intervention administration, otherwise the intervention effects are taken to be Gaussian noise, underestimating the actual autoregressive effect." If one uses the original acf/pacf rather than a acf/pacf conditional upon detected interventions then there often can be poorer results.

I took your 20 values and analyzed them with a robust approach which identified 4 pulses and an ar(1) arima component with only lag 2 .

Note Well that this is a totally different model from what has been represented here suggesting memory of order 2 and order 2 only not .16 and .7 respectfully but o. and .805 and the conditional mean is not 0.0 but -.05268 .

Here is the identified model in two views .enter image description here and enter image description here with model statistics here enter image description here

Some may bridle about using 6 coefficients for 20 observations but they should note that all coefficients are statistically significant yielding a necessary and sufficient model.

The Actual/Fit and Forecast are here enter image description here with forecasts converging to an assymptote enter image description here

EDITED after @whubers very insightful and thought-provoking questions.

When I ran it the first time I took the option of simply providing presumptive confidence limits based upon normality of the residuals. Here is the distribution of the residuals (hardly normal). enter image description here

This time I unleashed the monte-carlo bootstrapping option (2000 realizations per forecast period) where the residuals get re-sampled and limits are then based upon them. Furthermore I optionally enabled pulses to be present in the simulations thus 33.33% ( 4 out of 12 ) where you had correctly guessed 1 of 4 ) had pulses in the realizations because 4 pulses were found in the 12 (18-12) non-zero residuals.. 20 observations minus 2 for the ar and minus 6 for the estimated parameters enter image description here .In this way a more realistic assumption i.e. the possible re-occurrence of pulses in the future are put in place.

In this manner probability density functions can be constructed based upon the actual distribution of residuals without having to give that distribution a name.

enter image description here and the forecasts here enter image description here

Prof. Sam Savage https://www.probabilitymanagement.org/ was inspirational in motivating this particular developmemt of AUTOBOX to meet his simulation/analytical requirements as he wished to have realistic forecasting limits.

A table comparing standard normality-assuming limits(right-hand side of table ) versus the monte-carlo/simulation approach (left) is illuminating. enter image description here

As usual your thoughtful reflections motivate (TEASE OUT) more critical detail from me.


To discuss the differences between the two models , one simply has to compare the model, fit and forecast. I used your ar(2) model form and estimated parameters and obtained a somewhat surprisingly different and better solution in terms of estimated variance . enter image description hereenter image description here and here enter image description here with forecasts that are getting smaller as you suggested BUT are better characterized as converging to an assymptote that is lower than recent valuesenter image description here and are not truly going to get get smaller forever as intimated by your question.

Your models forecasts are here enter image description here with Autobox's forecasts here from it's automatically identified model enter image description here

| cite | improve this answer | |
  • $\begingroup$ I am impressed with the fit. But aren't "pulses" variables that would not otherwise have been predicted before they occurred? Your analysis appears to have found four such events out of 20. A crude but reasonable estimate is that they might continue to appear at a similar rate. I have long been curious about the role pulses might play in your forecasts. How is the possibility of future pulses incorporated in the forecast of 10 more observations, in which we would expect to see at least one pulse and as many as four? $\endgroup$ – whuber May 19 '19 at 21:10
  • $\begingroup$ @IrishStat I really appreciate your detailed response. Much more than I expected! I'm afraid since I'm a beginner to time series analysis I didn't understand a lot of what you posted though. I apologize for what's probably a stupid question but can you explain why the model was so different? I'm assuming your software specifies the order automatically? $\endgroup$ – Michael Howell May 20 '19 at 3:29
  • $\begingroup$ please see my edited response answering your question . Also yes this was an automatic model building excercise. $\endgroup$ – IrishStat May 23 '19 at 19:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.