0
$\begingroup$

I fit a statsmodels.tsa.statespace.sarimax.SARIMAX model (statsmodels==0.8.0) but I'm getting unexpected forecasting behavior, in which the forecast has a negative slope (see last plot at the bottom).

Below are my endogenous and exogenous data, which have hourly sampling frequency. The endogenous variable appears to have 24 hour season.

enter image description here enter image description here

Below is a time series diagnostic plot of the endogenous data. In the top figure, the red line is the rolling mean and the purple line is the rolling std:

enter image description here

After applying a season difference and first difference (e.g. using pandas, endog.diff(24).diff().dropna() I get a diagnostic plot like:

enter image description here

Which lead me to believe SARIMAX(0,1,0)(1,1,1,24) might be appropriate. This is the code I used to instantiate and fit the model:

sarimax = SARIMAX(endog=endog_tr, exog=exog_tr,
                  order=(0,1,0), seasonal_order=(1,1,1,24),
                  trend='n')
res = sarimax.fit()

Here is the result summary:

                                 Statespace Model Results                                 
==========================================================================================
Dep. Variable:                              endog   No. Observations:                 6547
Model:             SARIMAX(0, 1, 0)x(1, 1, 1, 24)   Log Likelihood                8437.861
Date:                            Fri, 30 Mar 2018   AIC                         -16867.721
Time:                                    23:53:46   BIC                         -16840.574
Sample:                                01-01-2017   HQIC                        -16858.335
                                     - 09-30-2017                                         
Covariance Type:                              opg                                         
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
exog          -0.0013      0.001     -1.499      0.134      -0.003       0.000
ar.S.L24       0.2415      0.009     26.204      0.000       0.223       0.260
ma.S.L24      -0.9139      0.005   -189.131      0.000      -0.923      -0.904
sigma2         0.0044   4.27e-05    102.700      0.000       0.004       0.004
===================================================================================
Ljung-Box (Q):                      290.33   Jarque-Bera (JB):              5668.80
Prob(Q):                              0.00   Prob(JB):                         0.00
Heteroskedasticity (H):               1.51   Skew:                            -0.11
Prob(H) (two-sided):                  0.00   Kurtosis:                         7.56
===================================================================================

Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).

After fitting the SARIMAX model, I did another diagnostic plot on the residuals:

enter image description here

I believe the data looks mostly stationary. There appears to be some seasonal correlations still, but I'm not sure how to get rid of that.

I did some in-sample plotting using res.predict(). The predictions appear to match the endogenous variable (labeled "target") quite well:

enter image description here

Now here's where things go wrong. I want to forecast several days out, but the forecast has an odd downward slope. Here's how I produce the forecast:

preds = res.forecast(exog_test.size, exog=exog_test.values.reshape((-1, 1)))

and here's the resulting plot, along with the ground truth test data:

enter image description here

Does anyone know why this is happening? I'd appreciate any help.

Edit:

I've added a notebook reproducing my work as well as some sample data:

Edit 2:

I've added more notebooks:

$\endgroup$
  • $\begingroup$ Did you check that exog_test.values has the correct numbers? The forecast trend is too regular to be driven by fluctuating exog. I don't see any other reason why this should happen. Can you provide a full working example to investigate? (A link to a notebook would be the most convenient.) $\endgroup$ – Josef Apr 1 '18 at 16:55
  • $\begingroup$ Hi @user333700, thank you for commenting and I appreciate your input. The exog_test values appear to be correct from what I can tell. Good idea, I will produce a notebook as well as the dataset in a flat file. It may take me a day or so due to other obligations. $\endgroup$ – trianta2 Apr 1 '18 at 21:32
  • $\begingroup$ @user333700 I've added a notebook and some sample data $\endgroup$ – trianta2 Apr 2 '18 at 21:41
  • $\begingroup$ I;m having the exact same problem, the predict looks a lot like the target and then the forecast is a straight line :( I just can't understand the difference between predict and forecast, documentation is just awful $\endgroup$ – Yuca Aug 16 '18 at 23:17
4
$\begingroup$

You are specifying an I(2) process, so you're specifying that the change in the time series is itself integrated. The forecast for the change of the series is then like a random walk (i.e. it won't die out). This estimate of this change (and so the forecast going forward) is encapsulated by the last estimated state (i.e. when the model is cast in state space form).

Because the change of the series is fixed (either positive or negative depending on the last estimated states), the forecast will trend up or down, regardless of the AR and MA coefficients.

Since the model is seasonal, there will also be a seasonal pattern, but the same general explanation applies to a non-seasonal ARIMA model with d=2.

It seems like you don't need seasonal differencing here - have you considered SARIMAX(0,1,0)(1,0,1,24)?

$\endgroup$
  • $\begingroup$ Hi @cfulton, I tried SARIMAX(0,1,0)(1,0,1,24) in the "First difference test" notebook I added (see cell In [6]), but the problem remains. The first difference ACF and PACF plots in In [4] made me think SARIMAX(1,1,0)(1,0,0,24) might be appropriate, but the prediction plot in Out[8] shows that the forecast quickly attenuates. $\endgroup$ – trianta2 Apr 5 '18 at 21:44
  • $\begingroup$ Actually, sorry, I think eliminating the non-seasonal differencing is better. I'm getting pretty reasonable results with SARIMAX(1, 0, 0)(1, 1, 1, 24) $\endgroup$ – cfulton Apr 7 '18 at 2:09
  • $\begingroup$ Indeed, SARIMAX(1, 0, 0)(1, 1, 1, 24) appears to work nicely and has removed the downward slope. Thanks! $\endgroup$ – trianta2 Apr 10 '18 at 15:55
1
$\begingroup$

Kudos on your presentation . The problem you are running into maybe due to the model that you are specifying . Hourly data is often quite dependent on daily activity which in turn can often be described (instantiate in your words) by deterministic effects. We have found that a two level approach can be useful Hidden markov model to detect Stock outs in Hourly sales Time series data where 24 hourly models are developed using as a predictor the daily totals.

It appears that your non-significant predictor may be driving the downward forecast but I would have to delve much further into your data to substantiate that reflection . I have also seen the inclusion of a non-significant trend constant can lead to your circumstance.

$\endgroup$
  • $\begingroup$ Hi @IrishStat, thank you for your answer. I'm less experienced with HMMs. To clarify, are you suggesting that I could use a HMM to predict latent state, and use that state as predictor for the hourly model? If so, thinking along those same lines, if days are correlated, do you think summary statistics of the previous day might help? Regarding your comment on non-significant predictor, I've tried removing the exog variable and the issue persists. I also believe my existing model was specified to not include a trend constant. $\endgroup$ – trianta2 Apr 3 '18 at 14:56
  • $\begingroup$ If summary statistics i.e. the previous day's total is important it will be incorporated as part of the prediction model for the daily totals and subsequently possibly impact the expected value for each hour. See stats.stackexchange.com/questions/162966/… for another discussion $\endgroup$ – IrishStat Apr 3 '18 at 15:19
  • $\begingroup$ Hi @IrishStat, I wanted to quickly evaluate your proposal of using a daily predictor, so I used the daily means from the dataset directly as a predictor in the hourly model. However, I'm not sure if this is a kosher approach to ceiling analysis in this scenario. The results are in the "Daily predictor test" notebook I added, and unfortunately the daily predictor did not help. $\endgroup$ – trianta2 Apr 5 '18 at 21:51
  • $\begingroup$ I am a little confused and monologues are not really helpful. Did you build 24 individual models ? Perhaps a skype session will clear up things $\endgroup$ – IrishStat Apr 5 '18 at 23:33
  • $\begingroup$ IrishStat, thank you for your generous offer of a skype session to help debug my issue, however cfulton's suggestion appears to have fixed my problem. I will continue to explore your two-level approach to improve my forecasts. $\endgroup$ – trianta2 Apr 10 '18 at 16:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.