2
$\begingroup$

I've got energy demand data and I'm trying to use ARIMA model to forcast future values but it doesn't look very good.This data has two seasonalities(daily and weekly) which I inculde in the model with xreg=cbind(z,zf) where z,zf are fourier transforms of my time series with given frequencies (24hz for daily and 24*7 for weekly)

my code in R

y <- ts(daily_data_forecast$real,frequency = 24)
z <- fourier(ts(daily_data_forecast$real, frequency=24*7), K=5)
zf <- fourier(ts(daily_data_forecast$real, frequency=24), K=5)
fit <- auto.arima(y, xreg=cbind(z,zf), seasonal=FALSE)
fc <- forecast(fit)

plot(fc)
lines(data)

enter image description here

blue line represents my forecast,red/black are original data/predicted data with unknown but better than my model

results of head(daily_data_forecast)

1 01.03.2018 01:00:00    20000 20037.60 01.03.2018 01:00
2 01.03.2018 02:00:00    19400 19471.28 01.03.2018 02:00
3 01.03.2018 03:00:00    19100 19239.34 01.03.2018 03:00
4 01.03.2018 04:00:00    19200 19243.39 01.03.2018 04:00
5 01.03.2018 05:00:00    19800 19547.79 01.03.2018 05:00
6 01.03.2018 06:00:00    21100 20260.26 01.03.2018 06:00

where I use columnt starting with 20037.60 as input of my model.

I would be grateful for any advice.

I've added link to my .csv data

$\endgroup$
6
  • $\begingroup$ You have mean shift, or a trend in the data. Nothing can catch that $\endgroup$
    – Aksakal
    Commented Apr 17, 2018 at 17:35
  • $\begingroup$ @Aksakal AUTOBOX routinely catches them autobox.com/cms/index.php/afs-university/intro-to-forecasting/… .. and my response here via Intervention Detection docplayer.net/… $\endgroup$
    – IrishStat
    Commented Apr 18, 2018 at 9:49
  • $\begingroup$ @IrishStat, I believe your training sample is much longer than in the OP's example, and that you allowed several periods with a new mean into a training set unlike OP $\endgroup$
    – Aksakal
    Commented Apr 18, 2018 at 10:47
  • $\begingroup$ I had asked him to provide (just the historical data prior to your forecast) but it appears you have very very good eyes. It looks like he posted two additional days (2 x 24 x 7 =236 additional values) thus I used 1128 values (47 x 24 =1128) wheres apples-2-apples should have been 892. $\endgroup$
    – IrishStat
    Commented Apr 18, 2018 at 11:08
  • $\begingroup$ As you correctly reflected even advanced level shift/time trend detection schemes wouldn't detect the downwards level shift because it was not present in the analyzed data set. In my opinion this example highlights the disingenuous feature of splitting the "historical data" as what occurred in the most recent 236 values was fundamentally unpredictable without additional information e.g. monthly effects. $\endgroup$
    – IrishStat
    Commented Apr 18, 2018 at 11:19

2 Answers 2

0
$\begingroup$

We have seen many examples of data like this where arima (memory) is woefully inadequate. In many cases ( if not most ! ) your are better served with a model that incorporates/mixes daily effects. hourly effects,level shift effects,time trend effects while dealing with/adjusting for anomalies AND arima. If you post your data in a csv file (just the historical data prior to your forecast) I will see if I can find time to help you further. You might want to search for previous postings of mine detailing this.

EDIT after receipt of data ..

Daily data is often strongly affected by calendar events as we are creatures of habit. Your data strongly suggests the need for both memory (arima) and hourly deterministic effects . Furthermore there is a significant downward deterministic trend and a number of anomalies in the data.

Following is plot of the Actual and the Forecast from the totally automatic modelling option . enter image description here . A busier picture is the Actual/Fit and Forecast with 95% limits enter image description here

The equation is presented here in two parts due to it's size ... The first portion are the hourly coefficients enter image description here while the second presents the trend and the anomalies portion AND the arima portion enter image description here . The model statistics are here enter image description here with the forecasts presented here enter image description here . The forecasts are partially listed here enter image description here

Note the arima coefficient suggests a random walk component which fundamentally means that in addition to the deterministic components (hour of the day , trend ) one needs to incorporate the previous value in the equation as .978 is nearly 1.0.

I would be very interested if you or some other responder/reader found a better representation of your data than what is presented here and if so it should be posted for all to see .

Day-of-the-week indicators/effects were found to be not-significant and thus are not part of the final model. I should also add that I have helped to develop the software used in this tour de force (AUTOBOX). Now that you know a useful model you should be able to replicate it's forecasts and make an assessment.

The answer to your question is to add 22 hourly dummmies , 2 time trend indicators and adjust/cleanse a number of unusual values which reflect unknown/unspecified exogenous effects.

The simultaneous effect of the (near) differencing operator and the trend variables culminates in what is effectively (nearly) a slightly declining level shift forecast with a slight decline. The forecast clearly captures prior images being launched from the low last value of 16495 reflecting the adaptive nature. If the last value was higher the forecast pattern would be shifted up proportionately.

$\endgroup$
1
  • 1
    $\begingroup$ Thank you for your answer.I've added my data in csv file as a link in my question.I will also look into yours posts,to seek more info about time series forecasting! $\endgroup$
    – wiedzminYo
    Commented Apr 17, 2018 at 16:33
1
$\begingroup$

ARIMA isn't suited for multiple seasonalities. In your example, you are using an $ARIMA(1,1,4)(1,0,0)_{24}$ model.

Since you are using multiple seasonalities, ideally you should be able to implement some sort of "double seasonal"

ARIMA model $ARIMA(1,1,4)(1,0,0)_{24}(p,q,d)_{168}$

since 168 hours is your weekly frequency.

But no such "double seasonal" ARIMA model exists. You might be able to approximate it as an $ARIMA(168,1,4)(1,0,0)_{24}$ model or some higher order model - but 168 is a ridiculously high order for an ARIMA model as is not recommended. Even the 4 in your non-seasonal component is unusually high, and it is likely what is causing those strong downwards oscillations in your forecast (informally it is not recommended that the orders go above 2 in an ARIMA model - although I don't have any references to back that up other than hearsay).

You would be better off using TBATS models, which are designed specifically for series with multiple seasonalities, and which are also easy to implement in R.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.