Folks, I am working on time series traffic data where the waiting times are indexed over time, with 288 observations for 24 hour time period (interval of 5 minutes). I am trying to cleanse the data, identify the patterns and create a forecasting model. (30 days of data)

Trouble is, 80% of the observations are marked as "No Delay", which means waiting time is 0 minutes and 20% of observations are continuous data where waiting time ranging from 3 minutes to 87 minutes.

I looked at multinomial probit regression and multinomial logistic regression and not sure if this is the right way forward. Logistic regression, because I am thinking in the lines of treating "No delay" as categorical variable but waiting times are actually continuous.

I am unable to look at ARIMA and the likes because, after I cleansed the data (removing duplicates and missing values) the intervals are not equally placed. I have data for 12:00 AM, 12:05 AM, 12:10 AM, 12:25 AM (15 minutes interval -- missing values found for 12:15 AM and no record exist for 12:20 AM) .

How to treat missing values here ? Is logistic regression the right approach? or Time Series models? if time series, how to work with unequal intervals?

Thanks for your effort in helping, grateful for feedback in constructing my question properly.

  • 1
    $\begingroup$ (1) Is it possible that "no delay" really means "less than three minute delay"? (2) What is the reason some cases, such as 12:15 am, are missing? Could it be that these tend to be associated with no-delay circumstances? $\endgroup$ – whuber Dec 1 '14 at 3:25
  • $\begingroup$ State space representations of ARIMA-like models have no issue with uneven time intervals. $\endgroup$ – Aksakal Dec 1 '14 at 3:40
  • 1
    $\begingroup$ @whuber: 1. That could be a good interpretation. You are right, it cannot be literally zero minutes. 2. Poor data collection. Do you suggest we could populate with some values instead of removing entirely from the dataset? $\endgroup$ – kpnane Dec 1 '14 at 14:52
  • 1
    $\begingroup$ I added some relevant tags (censoring, missing-data) to help attract knowledgeable readers and to cause useful threads to appear under the "Related" header at the right of this page. $\endgroup$ – whuber Dec 1 '14 at 15:04
  • $\begingroup$ @Aksakal I will explore that in SAS today. Since this data is in form of arrival process (not entirely), do you think Poisson regression would fit? $\endgroup$ – kpnane Dec 1 '14 at 22:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.