7
$\begingroup$

This is really puzzling...

I have this data which has a lot autocorrelations...

The data is about 60000 data points of 15min data.

I tried fitting it to ARIMA(6, 0, 6) and even GARCH(1, 1) with mean model ARMA(6, 6), still there are lots of autocorrelations in the residuals. enter image description here I almost wanted to try ARIMA(100,0,100), but I think even that is not enough...

I am doing these in R.

How to I get out this swamp? Please shed some lights on me.

Thanks so much

Update: I have fitted a gjrGARCH model with distribution "sstd" in R using "rugarch" package...

QQ-Plot of Standardized Residuals looks great - it's a straight line...

However the ACF plots of the residuals and the residuals squared are a bit strange, especially the big spike in the residual squared ACF plot.

Could anybody please shed some lights on us, esp about that big spike?

Thank you!

enter image description here

enter image description here

Improve this question
$\endgroup$
3
  • 3
    $\begingroup$ "how do I accept an answer on this website?" There are empty check marks under the posted answers to your questions. Click one to make it green. Here's a discussion on accepting answers on meta: meta.stackexchange.com/questions/5234/… $\endgroup$
    – jthetzel
    Commented Apr 28, 2012 at 21:55
  • 2
    $\begingroup$ Perhaps a better question is whether ARIMA is the appropriate model in the first place? $\endgroup$
    – Emre
    Commented Apr 28, 2012 at 22:14
  • $\begingroup$ @Emre: why do you think ARIMA is not suitable for this data? what other models are there? I was originally thinking of the route of "mean-model", "vol-model", etc... $\endgroup$
    – Luna
    Commented Apr 28, 2012 at 23:46

2 Answers 2

Reset to default
9
$\begingroup$

I think the issue here is whether a hypothesis test of the residuals is appropriate. You have 60000 observations, so any model will fail a residual test as there is so much data. That doesn't make the model bad, it just means that you have enough data to be able to tell that the model is an inaccurate representation of reality.

Step back and ask, what do you want a model for? And what do you know about the data that would help in selecting an appropriate model? Whatever model you end up with, don't expect to find that the residuals are white noise. With enough data, every model can be shown to be inadequate.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thank you Rob! I wanted a model for two purposes: (1) initially I wanted to estimate a vol model to replace the simplistic std estimate of volatility; (2) then I thought about using the mean-vol model to do forecasting... I have posted the GARCH vol-model part here: stats.stackexchange.com/questions/27319/… $\endgroup$
    – Luna
    Commented Apr 29, 2012 at 18:39
  • 1
    $\begingroup$ If I cannot expect the residuals to be white noise, then how do I do model-comparison and model-checking? How do I know if, for the mean-model, ARFIMA(1, 0.5, 1) is better than ARFIMA(2, 0, 2), etc.? Thanks again! $\endgroup$
    – Luna
    Commented Apr 29, 2012 at 18:41
  • 1
    $\begingroup$ Use the AIC to select a model. $\endgroup$
    – Rob Hyndman
    Commented Apr 29, 2012 at 23:24
  • $\begingroup$ So I　ｓｈｏｕｌｄ　ｉｍｐｏｓｅ　ａ　ＧＡＲＣＨ(４０,　１)　ｍｏｄｅｌ，　ａｓ　ｓｈｏｗｎ　ｆｒｏｍ　ｔｈｅ　ｒｅｓｉｄｕａｌ　ＡＣＦ　ｐｌｏｔ　ｏｆ　ｔｈｅ　ｒｅｓｉｄｕａｌｓ　ｓｑｕａｒｅｄ　ａｂｏｖｅ？　Ｔｈａｎｋｓ！ $\endgroup$
    – Luna
    Commented Apr 30, 2012 at 3:17
5
$\begingroup$

We are working with data like this for a major fast food franchise. The series represents the demand for tacos in 15 minute intervals for the last 5 years (180,000 observations) . This series can be treated by building 96 separate models (4x24) for each 15 minute interval a daily model reflecting overall trends,level shifts,holiday effects etc in daily values. By integrating the impact of daily values and their history on each of the 96 models and then reconciling, we are able to accurately predict both the demand for 15 minute intervals and the daily totals. The reason you think the acf is significant is as Rob points out due to the sample size since the standard error of the acf is equal to 1/sqrt(N).

@Luna As you correctly point out in your comment one loses the connection between the different time slices BUT one gains the impact of activity over days/weeks/months while being able to detect changes in daily effects , while discovering the impact of particualr days of the month etc.. We like you had studied the "one-time series approach" using semi-hourly electricity demand data only to conclude that we were getting FALSE CONCLUSIONS due to the size/length of the data. In general one could have 96 equations with X eXogenous series . This would be called a Vector ARIMA problem and would be unwieldy as outlier /inliers cpuld distort parameter estimates. Standard errors would be microscopic in size due to large N . We have found ways to incorporate daily trends directly into each of the 96 equations

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thank you Dave! So you have 5 years of 9:00am data, 5 years of 9:15am data, 5 years of 9:30am data,... these are all daily time series, and you fit these daily time series (there are 96 of them)? this approach is interesting, but then you neglect the connections between the 9:00am and the 9:15m and the 9:30am data points, no? Is there a way to do, as you suggested, the separate model fitting for 96 of your daily time series, and in the meantime, model the interrelation among these 96 models? Thank you! $\endgroup$
    – Luna
    Commented Apr 29, 2012 at 18:46
  • 1
    $\begingroup$ @Luna As you correctly point out one loses the connection between the different time slices BUT one gains the impact of activity over days/weeks/months while being able to detect changes in daily effects , while discovering the impact of particualr days of the month etc.. We like you had studied the "one-time series approach" using semi-hourly electricity demand data only to conclude that we were getting FALSE CONCLUSIONS due to the size/length of the data. In general one could have 96 equations with X eXogenous series . This would be called a VeEctor ARIMA $\endgroup$
    – IrishStat
    Commented Apr 29, 2012 at 19:54
  • $\begingroup$ Do you have a reference with an example of this type of model? I am particularly interested in how you deal with price(s) and what the exogenous variables are. $\endgroup$
    – dimitriy
    Commented Apr 30, 2012 at 15:21
  • 1
    $\begingroup$ @Dimitri We have just developed this enhanced feature within AUTOBOX , a software package that I helped to develop. Our customer a large fast food franchise will not allow us to publish the results as they don't want their competitors to be able to work/plan smarter. I can tell you that the system allows for price/promotion/holidays etc to play a role in both the 96 individual forecasts and the daily forecast. We are using promotional inputs as predictor series. If you want them to demo this for you please contact them at [email protected] $\endgroup$
    – IrishStat
    Commented Apr 30, 2012 at 21:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.