I'm looking at attempting to capture the regularity of a time series of events, one measurement per day, with a year's worth of data, and there can be at most one event a day. Say for example the day you do laundry. What I want to capture is a measurement of regularity. Capturing irregularity is straight forward: goodness of fit of the times between consecutive events with a poisson distribution. But this doesn't distinguish between poor fitting series very well.

So I want to measure the other end. How good am I to sticking to a weekly or bi-weekly schedule. My instinct is I want to fit an autoregressive model and take the variance of epsilon. Does this sound right? And how do I normalize for frequency? That is I don't want the epsilon to predict whether I do laundry 50 times a year or 100 times. Just the regularity.

Or is there a conventional way of doing this that I'm overlooking? In this case I'm biased towards fast conventional techniques rather than creative techniques.

Thanks! Steven

  • $\begingroup$ hmmm.. an auto-regressive model isn't really a good fit for a binary time series is it? Auto-regressive on time between events? I hate the idea of variance of time between events. $\endgroup$ – Steven Noble Nov 10 '10 at 21:09

Let $X_t$ be the time between events. Then for very regular events, $X_t$ will be approximately constant. e.g., if you do your laundry every Monday, then $X_t=7$ for all $t$. So you could just use the variance of $X_t$, where the small variance corresponds to highly regular and large variance corresponds to low regularity.

If outliers are a problem, or you do not want occasional "misses" to affect the result, use the interquartile range (IQR) instead of the variance. If that removes too many observations, try the difference of the 90th and 10th percentiles.

To normalize, divide by the median (safer than assuming you know the frequency). Thus, one possible measure which should work for you is:

$$ (q_{0.9} - q_{0.1})/q_{0.5} $$ where $q_{\alpha}$ denotes the $\alpha$-quantile.

  • $\begingroup$ Thanks Rob! I do want to note that I added my question comment before your answer so there is no way it was intended as an insult to your generous answer. I have 2 hesitations about variance of time between events (though I may go with it). There's the general "does this assume too much normality." But my bigger concern is someone who does laundry twice a month, but misses a few (maybe due to travel) will have a series like "7,7,14,7,7,7,21" which would have a var similar to a more frequent but less regular individual. What do you think of multiplying the std dev by the frequency to norm'ze? $\endgroup$ – Steven Noble Nov 11 '10 at 5:24
  • $\begingroup$ oh... can we use Latex here? $\endgroup$ – Steven Noble Nov 11 '10 at 5:26
  • $\begingroup$ Variance does not assume normality. It is a general measure of spread. If you prefer a more robust measure that will allow for the occasional misses, uses the difference of two percentiles. I don't understand why you might want to multiply a measure of spread by the frequency. That would not normalize it. I've updated my answer. $\endgroup$ – Rob Hyndman Nov 11 '10 at 5:45
  • $\begingroup$ Of course the measurement itself doesn't assume normality, but if you are using it as a comparison of inner n-tile ranges you are at least assuming the distributions are of similar forms; if not normal. But yes, that's not a big deal. If you have 2 sets of diffs {6,6,6,12,18} and {2,2,2,4,6,2,2,2,4,6,2,2,2,4,6} you get standard deviations respectively of 1.66 and 5.37. Even though their regularities are similar. But multiply by washes/week you get 3.62 and 3.91. And these values will converge as you extend your sample period. Thank you for your further update. $\endgroup$ – Steven Noble Nov 11 '10 at 7:16
  • $\begingroup$ Multiply by the frequency is just another way of saying divide by the mean of the differences, right? $\endgroup$ – Steven Noble Nov 11 '10 at 7:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.