# Why ARIMA/ARMA is performing very bad on out of sample (future) prediction?

I am working on a time series forecasting problem. The time series is -

I plotted the rolling mean and rolling variance - rolling mean -

and the rolling variance-

The rolling mean/variance has not that specific pattern(increasing or decreasing).I used augmented Dickey–Fuller test and the p value is 0.0008.(I assumed that the I don't need an integrative component).

Here's the partial auto correlation plot -

Since only 1st and 4th lag seems significant ,I tried searching for the best parameter for ARMA but prediction on the out of future dates is quite strange. Here's the plot on the training data -

which gets worse on the future points and quite similar if I use ARIMA with I equals to 1-

Fit on seasonal Arima (1,0,0)(0,0,0)7-

I want to predict the future trend in this time series ,but due to very poor performance I am not sure how to proceed. I am a beginner in time series analysis ,please correct me if I have done some conceptual mistake.

My specific question is how to model the trend for future points in a time series if the time series model is not a good fit ?

• why don't u [post your data and I will try and help .. how many points did u withhold ? – IrishStat Mar 16 '19 at 15:26
• You asked "My specific question is how to model the trend for future points in a time series if the time series model is not a good fit ? ..". I answer in the spirit of @Stats ...Then identify an appropriate/useful model via diagnostics – IrishStat Mar 16 '19 at 15:41
• how many points did u withhold ? If i am right you are asking about the no. of test points , I have taken 20 percent data for test. – A.kumar Mar 16 '19 at 18:26
• Please post your data – IrishStat Mar 16 '19 at 21:04

I took your 412 daily historical values an introduced them to AUTOBOX. After some interrogation they disclosed that not only was there memory in the data : ARIMA (1,0,0)(0,0,0)7 but that latent deterministic structure was found/detected .

The deterministic structure found was 2 significant months were higher (August and September) while the first day-of-the-week was significantly lower (curiously a Tuesday ! ).

In addition there were three level shifts (periods 42, 273 and 317 now visually obvious !) and some unusual one-time pulses.

The reasons that you din't discover this model was 1) you and your software of choice limited your analysis to memory only i.e. ARIMA and more importantly your tool of choice expressly ignored deterministic effects which are often predominant in daily data due to important human patterns or habits or some unknown possibly unspecified persistant effect.

Here is the model and the Actual/fit and forecast graph nicely projecting into the future 103 days. Here are the forecasts for your 112 days that you withheld from the analysis

The plot of the residuals suggests randomness which is supported by the acf of the residuals

Finally the Actual and Cleansed graph visually presents the Identified Interventions

In summary 1) no variance change(s) was detected 2) no parameter change(s) were detected . Note well that if you don't deal with level shifts/changes you can falsely accept the hypothesis that are error variance changes.

Finally there are 3 types of time series models

1) autoprojective using only the history

2) deterministic using fixed dummies ( pulses,level shifts, seasonal pulses,local time trends

and

3) models that integrate both 1) and 2) ...like yours !

Hope this helps you , your fellow students , and your instructor , and of course other readers of SE ...

For @Stats no limits on forecasts and no negative forecasts.....

@Stats asked for a graph showing constrained forecast and forecast limits .

The OP asked for direction as to how they could implement the solution I presented. One way is to acquire acquire AUTOBOX ..there is an R version with major discounts to Universities. AUTOBOX is the only program I am aware of that simultaneously optimizes/identifies both the minimally sufficient ARIMA structure and the statistically significant Intervention Detection structure for both a Univariate case and a Multivariate case while dealing with non-constant model parameters and non-constant error variance..

If you want to build such model or close to that( a model with memory and latent deterministic changes) , I can suggest the following articles;

• Time series of interest seem to be positive. I looked at your forecasts and I saw many values of the lower forecast limit are negative. I also wonder how you control the effect (increasing number of false positives) of running many hypotheses at once. We know that "if you torture the data long enough, it will confess to anything" – Stats Mar 17 '19 at 20:40
• The limits are based upon re-sampling (bootstrap) the model residuals and using any variance inflation factor that may be relevant via the ARIMA model ( the psi weights) . There are no negative forecasts see additional graph .You are very right that one might want to tighten the alpha value . This is a user option. You don't have to fret about over-fitting as sequential tests of significance and necessity are in play . There is no torture here but intelligent step-forward (slowly) and verification of necessity (stepdown) procedures in play. – IrishStat Mar 17 '19 at 20:46
• The hypothesis are not "run at once" but are generated/ proferred when evidence supports them. Users can restrict the # of interventions found , their type and the level of confidence that must be met for model augmentation. AUTOBOX is not Torquemada just a skilled interrogator. – IrishStat Mar 17 '19 at 21:08
• @IrishStat thanks a lot for detailed explanation.Your model is integrating auto projective (ARIMA (1,0,0)(0,0,0)7 ) and effect of deterministic structures. I want to reproduce the results in python/R and I started with a Seasonal ARIMA model with the same coefficients((1,0,0)(0,0,0)7) but the model is still very bad on out of sample(image attached in question) because even now I am not taking the deterministic structural changes into account. Can you please tell the steps to proceed for this part/share some resource regarding that ? – A.kumar Mar 17 '19 at 22:33
• Thanks for your messages. On i.stack.imgur.com/fhpqP.png I see negative lower limits. How do you adjust you model so as to take into consideration this TS is positively valued? An hypothesis suggested by the data is likely to be one that has ‘stood out’ for some reason, and is likely to be accepted unless the bias is corrected for. How do you correct this bias? – Stats Mar 17 '19 at 23:16

What we can see on your data is

• Mean is time varying
• Variance is time varying (I don't understand how you came to the conclusion that the variance is almost constant)

Hence, look at the assumptions behind he ARMA models and ask yourself: "are those assumptions satisfied?". I believe one of these is the stationarity assumption.

Hence, your got your answer. We expect rubbish forecasts when we fit an inappropriate model to our data.

"how to model the trend for future points in a time series if the time series model is not a good fit ?"

Obviously further assumptions have to be made and tested as the ARMA ones don't appear to be plausible for your problem. One of these could be the "local stationarity" assumption. Could this assumption possibly be satisfied? If yes, you may choose (or build) a model among the class of models satisfying this assumption.

You basically have to identify attributes of the series that seem fairly stable in time and/or models consistent with the features of your series. And to be clear, I just give you some hints that may help you with any future research, as this is not a task for a beginner in time series analysis. Finally, note that quite often we fail miserably in detecting trends (e.g. especially when it comes to financial data), had that been easy, most of us would have been billionaires.

• quite often we fail miserably in detecting trends - Okay ,but can we just say that a time series is trend stationary if the p value of kpss test is far less than 0.05 ? – A.kumar Mar 17 '19 at 6:30
• Definitely not when the p value of KPSS test is far less than 0.05. – Stats Mar 17 '19 at 13:56
• Ohh ! sorry actually I wanted to write -can we definitely say that a time series is not trend stationary if the p value of kpss test is far less than 0.05. btw you answered the question. – A.kumar Mar 17 '19 at 14:00
• Answer which question? – user54285 Mar 18 '19 at 22:53