How to deal with interruption in time series analysis?

Question

I am probing a time series data of transactions. Basically, I want to see the pattern of the number of transactions in each time slice.

First of all, I looked at hourly data. However, the opening hour is 9am to 9pm thus no data in another 12 hours in each day.

Secondly, I looked at daily data (i.e. by date). No transaction occurred on Sunday and holiday so the data does not contain these date.

---

I read a book saying that:

One of the assumption of time-series analysis is data points are taken at equally spaced time steps, with no missing data points.

So should I fill the time (in hourly data) or date (in daily data) and interpolate the values (i.e. the number of transactions) to fulfill the assumption?

Answer 1

Definitely don't interpolate the transactions! That would be equivalent to saying you're doing business on Sunday at 6 AM. Just take those time points out; since we know for certain there cannot be any transactions outside of business hours, then we know there's no statistical information. Also, if there are any transactions outside of the appropriate hours, it must be a bug in the system or an unscrupulous employee doing some shady business by night.

The book passage that you read needs to be taken with a grain of salt; the author probably assumed that the data you got was processed in a logical manner to remove any excess stuff you didn't need. For example, if I have a data set which lists smartphone sales over the last 100 years, I'm definitely going to cut off anything before, say, 2000, because I am quite certain that nothing that we would call a 'smartphone' today existed back then on the market. This is a similar sort of data transformation.

For a more practical reason why you should take out the non-business hours data and then stitch together the remaining pieces, consider the following time series analysis problem. If we were to construct an autoregressive model of transactions, then there would be enormous error for the model in predicting the first hour of the day, as the preceding 11 time steps would all be zero. There's no way to add together multiples of zero to create a nonzero sum! Unless your ARMA includes a constant, that is.

There's also a bit of a problem with investigating this time series on an hourly basis since there is discontinuity between sales at 9PM and 9AM. Maybe something interesting can come out of analyzing it, but a more appropriate way to phrase this question is to look at each day as a member of a population of days with transaction data and then ignore the fact that each day is ordered. I say this because I would expect there to be little effect (on an hourly basis!) of something that happened on the preceding day. Unless, of course, there's that one really smelly employee who checks in, leaves a foul odor which then decays slowly over the next few days with regards to time.

TLDR: Don't interpolate! Either take out the offending hours and stitch the remaining data together, or consider each business day of transactions as a separate entity and compute population statistics on that group. Also, if you take out the holidays then the daily data should be perfectly fine to work with.