2
$\begingroup$

I have no background of advanced stats. I am an engineer and I have the following data. I am representing it as a decent graph for better understanding. I want to forecast the collision for the next hour with 95% confidence level in one hour interval.

enter image description here The x-axis is the time (total 7 hours) and the Y-axis is the number of collisions (in thousands). The unit time is 1hr. This is a time series data and the process is stochastic. The nature is somewhat random as there is a trend clearly be seen but sometime abrupt movement can also happen. I was wondering if I get the idea where to start to get a initial prediction. I was reading the Bayesian time series analysis and then moved to ARIMA model but I am confused. So, any help. I would also want to know the real research issues in these kind of data. The tool I will be using is R or might be Matlab.

$\endgroup$
5
  • $\begingroup$ Isn't y-axis the number of collisions? And what's the unit of time, e.g. that spike to 55k is 55k within what time interval? $\endgroup$
    – zbicyclist
    Commented Nov 26, 2014 at 16:17
  • $\begingroup$ Sorry for the typos. I already fixed it. You can consider 2 rectangular blocks equal to 1 hr. $\endgroup$
    – Kal
    Commented Nov 26, 2014 at 16:24
  • 1
    $\begingroup$ As an engineer, you would appreciate Transfer function modeling which is a more general class of models than ARIMA. Try searching "Transfer Function Time Series" "Interrupted Time Series". Although, I have never seen it implemented in a low frequency series such as yours. $\endgroup$
    – forecaster
    Commented Nov 26, 2014 at 16:36
  • $\begingroup$ @forecaster I am personally familiar with many successful applications of your suggested approach. $\endgroup$
    – IrishStat
    Commented Nov 26, 2014 at 21:30
  • $\begingroup$ @forecaster Actually this is the difference with the other approach. We can extend the time span (max of a week span) but we want it to predict it in hourly sense.. $\endgroup$
    – Kal
    Commented Nov 27, 2014 at 8:42

1 Answer 1

2
$\begingroup$

Your data set is similar to the thousands of hourly series that we have seen. Hourly predictions can often rely on previous hourly values (your ARIMA) but also on day-of-the-week effects , month(week)-of-the-year effects , holiday effects (both on and before/after the event) , Fridays before a holiday , Mondays after a holiday, specific days of the month , level shifts, local time trends , changes in parameters/error variance over time. All of this has to be done in a robust manner where unusual observations are identified and their effect nullified. I suggest that you address your problem with adequate methodology and useful software AND perhaps the advice of a trained time series statistician. Pursue robust Transfer Function identification also known as Dynamic Regression.

$\endgroup$
4
  • 1
    $\begingroup$ you correctly have identified the possible effects of the incidents. We are more on research side so we want to use some kind of opensource tool. Regarding the data based on the requirement I can get the data but short hour prediction what we need right now. We do not have any statistician in our team... $\endgroup$
    – Kal
    Commented Nov 27, 2014 at 8:38
  • $\begingroup$ Fair enough ... Use the opensource tool that delivers results similar to autobox.com/pdfs/capable.pdfs/capable.pdf ...particularly look at slides 44-55 as it details what you need to do. You might want to try and reproduce their results but if you don't have a statistician on hand that might be dicey. Bye the way I had helped develop some of AUTOBOX. $\endgroup$
    – IrishStat
    Commented Nov 27, 2014 at 14:08
  • $\begingroup$ I should also add that that this can also be accomplished at the hourly level thus incorporating daily predictions . If you wish to post your data or send it to me I will be glad to show you what you need to do or could do. . $\endgroup$
    – IrishStat
    Commented Nov 27, 2014 at 14:36
  • $\begingroup$ Thank you very much for the slides and the help. I will try to use some opensource tool (if I got any) for the data. If I stuck, I will get back to you :) $\endgroup$
    – Kal
    Commented Nov 28, 2014 at 8:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.