7
$\begingroup$

I have time series data that looks something like this:

    Series 1                         Series 2
    ╔══════╦════════╦════════╗       ╔══════╦════════╦════════╦════════╗
    ║ Time ║ Value1 ║ Value2 ║       ║ Time ║ Value1 ║ Value2 ║ Value3 ║
    ╠══════╬════════╬════════╣       ╠══════╬════════╬════════╬════════╣
    ║ 3:30 ║ 10     ║ 100    ║       ║ 3:32 ║ 12     ║ 56     ║ 34     ║
    ║ 3:31 ║ 11     ║ 50     ║       ║ 3:33 ║ 15     ║ 200    ║ 89     ║
    ║ 3:36 ║ 12     ║ 80     ║       ║ 3:35 ║ 15     ║ 90     ║ 66     ║
    ║ 3:50 ║ 11     ║ 80     ║       ║ 3:38 ║ 13     ║ 85     ║ 45     ║
    ║ 3:50 ║ 12     ║ 60     ║       ║ 3:45 ║ 14     ║ 65     ║ 121    ║
    ╚══════╩════════╩════════╝       ╚══════╩════════╩════════╩════════╝

The important features are

  • Data doesn't exist for every minute.
  • There can be multiple observations per minute.

I want to use a RNN to predict the next observation of Series 1 based on the observations of Series 1 and Series 2 up until that moment (and using keras specifically). The problem is that in all the RNNs I've found, the time interval between datapoints is constant. For example in an RNN that predicts the next character in a sentence, data in fed in 20 characters at a time, where the "time interval" between characters is constant.

So how can I adapt this to the data I have? My options seem to be:

  • Fill in the data for every minute, and average the observations if several of them appear in one minute. Then I could just use the data like the input to a character prediction RNN. The above data would become

    Series 1
    ╔══════╦════════╦════════╗
    ║ Time ║ Value1 ║ Value2 ║
    ╠══════╬════════╬════════╣
    ║ 3:30 ║ 10     ║ 100    ║
    ║ 3:31 ║ 11     ║ 50     ║
    ║ 3:32 ║ 11     ║ 50     ║
    ║ 3:33 ║ 11     ║ 50     ║
    ║ 3:34 ║ 11     ║ 50     ║
    ║ 3:35 ║ 11     ║ 50     ║
    ║ 3:36 ║ 12     ║ 80     ║
    ║ 3:37 ║ 12     ║ 80     ║
    ║ ...  ║        ║        ║
    ║ 3:50 ║ 11.5   ║ 70     ║
    ╚══════╩════════╩════════╝
    
  • This way really isn't ideal because quite some information is contained in the fact that multiple observations occur in one minute. It'd also substantially increase data size and training time.

  • Input the "Time" variable into the RNN as well, together with a "time since the last observation" variable. I'd also have to combine the two series into one, together with a flag (1 or 2, as a one-hot encoding) indicating which of the two series it came from. Then I can just feed the data in 20 (or whatever number) observations at a time, instead of 20 minutes at a time. I'm not sure that this would work very well, and I can't find anything like this on the internet. Is this a valid approach?
$\endgroup$
  • $\begingroup$ A solution to sparse time series is to "smooth" it. You can apply a localized regression algorithm to interpolate the "missing" values so that you have a complete data set with equal time intervals. $\endgroup$ – Jon Oct 25 '17 at 15:49
1
$\begingroup$

This depends a bit on whether interpolating between points to infill missing data points makes sense, i.e. is there a high correlation between points or are they completely random at each minute reading. If they are correlated then filling them in with even time intervals is probably the way to go. If the values are random and independent of each other, then this would not work.

The second method would (probably) not work, one issue would be that the time itself does not really impact the values directly, so you would have an input without predictive power in your RNN. This tends to produce poor results. The other large issue is to get your RNN to produce time points in the future; ANNs typically don't produce values outside of the observed range of input data.

If the interpolation method doesn't work for your time series data, I might recommend resampling at a coarser frequency, for example every 5 or 10 seconds, and grab the closest point you can at each of those intervals (or average if needed). It would reduce some information but would avoid potentially interpolating where it doesn't make sense.

In short this is a non-trivial question and as far as I know there is no standard or universal way of handling the issue of variable time increments, but hopefully these are some helpful thoughts.

$\endgroup$
  • $\begingroup$ There is some correlation between points and filling them out would somewhat make sense, but I really want to leave that as a last resort. And the data isn't really missing, it's just recording events that don't necessarily occur every minute, like when a red vehicle drives past and how many wheels it has. $\endgroup$ – ndrue Oct 20 '16 at 7:31
  • $\begingroup$ Thing is that time does effect the observations. For example there might be more red cars in the morning, or a red motorcycle (having 2 wheels) might suddenly become very likely after 2 red cars in the same minute followed by 3 minutes of no observations. $\endgroup$ – ndrue Oct 20 '16 at 7:35
  • $\begingroup$ In that case perhaps adding the time would make sense. I am not sure what the data is but it might make sense to transform the data to be representative of periods of the day for example, to have those impacts in your model. The other thing you should consider is having autoregressed data in your inputs, such as number of red cars in last time gap, or the actual time gap one, two, etc. points ago. Might add some predictive power to the model $\endgroup$ – Rob Oct 20 '16 at 19:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.