4
$\begingroup$

Thanks all I have edited the questions based on your replies

I have a univariate time series with high frequency (the data is generated minutely) with 10 days of data so far.

The data is coming from sensors located on different parking sites that count vacancy availability, my objective is to do a forecast for each parking site for the next 24 hours.

> str(stations)
'data.frame':   76234 obs. of  3 variables:
 $ siteid    : Factor w/ 5 levels "15031","15032",..: 3 4 1 5 2 3 4 1 5 2 ...
     $ Count     : int  13 16 30 2 9 13 16 30 2 9 ...
 $ LastUpdate: POSIXct, format: "2015-08-27 06:31:00" "2015-08-27 06:31:00" "2015-08-27 06:31:00" "2015-08-27 06:31:00" ...
> head(stations)
  siteid Count          LastUpdate
1  15033    13 2015-08-27 06:31:00
2  15034    16 2015-08-27 06:31:00
3  15031    30 2015-08-27 06:31:00
4  15035     2 2015-08-27 06:31:00
5  15032     9 2015-08-27 06:31:00
6  15033    13 2015-08-27 06:32:00

I have read through forums that arima models are not suited for high frequency data, when I decompose the ts that initiate with a frequency of 24*60.

> s15033<- stations[stations$siteid == 15033,]
    > s15033.ts <- ts(s15033$Count, start = c(2015,239), frequency = 1440)

Thank you for the reply

Here is an updated hourly sample of the dataset as requested

What would be the right approach and recommended method to use for such high frequency?

Edit

Thank you so much for your input, I had to digest all of the information you provided.

In my case, I have no choice but to use R so as that incoporate the model into the workflow of my application.

I understand that I will be better off, forecast at the hour level and use external regressors to help with the shifts and sudden spike that may appear in the time series.

As follow up, I have a couple of questions and apologies if the questions may sound a bit rudimentary as I am still learning about forecasting and R

  • How can I implement in R the hybrid causal/ARIMA/pulse you came up with in Autobox
  • I have about 1000 parking site, would I need a model for each parking site
  • Would machine learning model help with predicting at lower granularity
Improve this question
$\endgroup$
6
  • $\begingroup$ which language are you using? can you make your example reproducible by providing some sample data? $\endgroup$
    – Antoine
    Sep 6, 2015 at 17:01
  • 1
    $\begingroup$ I gather you have 10x24x60 = 14400 observations why don't you post your data and we will see what models/forecasts can be generated by the list. $\endgroup$
    – IrishStat
    Sep 6, 2015 at 17:02
  • 1
    $\begingroup$ What kind of data is it? The nature of the process that creates data is important when selecting forecast method. $\endgroup$
    – Ho1
    Sep 6, 2015 at 17:08
  • 2
    $\begingroup$ The data is the data and it's characteristics are what is important in selecting the right forecasting method. If there is an accepted underlying theory then this can often be used to efficiently form a useful/starting model model as compared to drawing/identifying the model from the data. $\endgroup$
    – IrishStat
    Sep 6, 2015 at 17:13
  • $\begingroup$ Well what did you think of my answer ? $\endgroup$
    – IrishStat
    Sep 9, 2015 at 12:54

1 Answer 1

Reset to default
1
$\begingroup$

You have data that is recorded every minute (60) for 24 hours for 10 days for 5 sites. Just because you have readings every minute doesn't mean you need to forecast at that level of granularity. It complicates the analysis significaantly due to the three possible levels of seasonality (minute/hour/day). I suggest that you initially forecast at the hourly level thus significantly reducing the analytics. In this way your data csv file could contain readings for the hourly values i.e. 24 per day (:00 readings). Time series analysis leans heavily/requires one having readings at each and every period and care must be taken to verify that. I found that your csv file seemed to have a flaw in that regard as there was no record for hour 2:00 for day 8/30 thus any analysis of the time dependent structure would be flawed. There may be other occurences of this so please check your data carefully enter image description here . Please resubmit a corrected csv file and I will attempt to model hour-of-the-day and day-of-the-week effects taking into account any level shifts and/or outliers. Since you only have 10 days any holiday effects would be not-identifiable. Normally in cases like this we suggest 3+ years of daily observations at the hourly level in order to assess holiday event activity. If other activity is known to have occurred (special one-off variables) then the user should include that information to keep the software free of unwanted outlier detection. Since this is a demonstration of what should/could be done you can limit your data set to just 1 site to illustrate the art-of-the-possible and how you might proceed.

EDITED AFTER RECEIPT OF CORRECTED DATA:

The data set contains 145 values (7 on day 0 and then 11 full days of 12 hourly values starting at day1/AUG28/Friday followed with 6 values for day12/Tuesday ). The first 7 values were discarded as being part of a day thus the 138 vales reflect 12 readings for 11 days plus 6 for today Sept 8/Tuesday was used. Thus the data used started at hour 1. A scan of the historical data enter image description here suggested either a level shift lasting two days (observations 49-72) or a Tuesday/Wednesday effect (day 5 and 6). It is impossible just having 11 complete days to conclude either way. On the off chance that this was a Tuesday/Wednesday effect rather than a step/level shift , AUTOBOX ( my software of choice ) was set up to consider daily effects enter image description here and to automatically develop a model and to predict out the next 24 hours (6 for today(Tuesday/12 for tomorrow(Wednesday)/6 for (Thursday). Here is the Actual/Fit/Forecast graph enter image description here . The Forecasts are here enter image description here and here enter image description here

The model is a hybrid causal/ARIMA/pulse detected example ..here enter image description here and here enter image description here and here enter image description here . The plot of the residuals suggests randomness enter image description here and is confirmed with the acf of the residuals enter image description here

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.