# RNNs for Sparse Time Series Data

I have time series data that looks something like this:

    Series 1                         Series 2
╔══════╦════════╦════════╗       ╔══════╦════════╦════════╦════════╗
║ Time ║ Value1 ║ Value2 ║       ║ Time ║ Value1 ║ Value2 ║ Value3 ║
╠══════╬════════╬════════╣       ╠══════╬════════╬════════╬════════╣
║ 3:30 ║ 10     ║ 100    ║       ║ 3:32 ║ 12     ║ 56     ║ 34     ║
║ 3:31 ║ 11     ║ 50     ║       ║ 3:33 ║ 15     ║ 200    ║ 89     ║
║ 3:36 ║ 12     ║ 80     ║       ║ 3:35 ║ 15     ║ 90     ║ 66     ║
║ 3:50 ║ 11     ║ 80     ║       ║ 3:38 ║ 13     ║ 85     ║ 45     ║
║ 3:50 ║ 12     ║ 60     ║       ║ 3:45 ║ 14     ║ 65     ║ 121    ║
╚══════╩════════╩════════╝       ╚══════╩════════╩════════╩════════╝


The important features are

• Data doesn't exist for every minute.
• There can be multiple observations per minute.

I want to use a RNN to predict the next observation of Series 1 based on the observations of Series 1 and Series 2 up until that moment (and using keras specifically). The problem is that in all the RNNs I've found, the time interval between datapoints is constant. For example in an RNN that predicts the next character in a sentence, data in fed in 20 characters at a time, where the "time interval" between characters is constant.

So how can I adapt this to the data I have? My options seem to be:

• Fill in the data for every minute, and average the observations if several of them appear in one minute. Then I could just use the data like the input to a character prediction RNN. The above data would become

Series 1
╔══════╦════════╦════════╗
║ Time ║ Value1 ║ Value2 ║
╠══════╬════════╬════════╣
║ 3:30 ║ 10     ║ 100    ║
║ 3:31 ║ 11     ║ 50     ║
║ 3:32 ║ 11     ║ 50     ║
║ 3:33 ║ 11     ║ 50     ║
║ 3:34 ║ 11     ║ 50     ║
║ 3:35 ║ 11     ║ 50     ║
║ 3:36 ║ 12     ║ 80     ║
║ 3:37 ║ 12     ║ 80     ║
║ ...  ║        ║        ║
║ 3:50 ║ 11.5   ║ 70     ║
╚══════╩════════╩════════╝

• This way really isn't ideal because quite some information is contained in the fact that multiple observations occur in one minute. It'd also substantially increase data size and training time.

• Input the "Time" variable into the RNN as well, together with a "time since the last observation" variable. I'd also have to combine the two series into one, together with a flag (1 or 2, as a one-hot encoding) indicating which of the two series it came from. Then I can just feed the data in 20 (or whatever number) observations at a time, instead of 20 minutes at a time. I'm not sure that this would work very well, and I can't find anything like this on the internet. Is this a valid approach?
• A solution to sparse time series is to "smooth" it. You can apply a localized regression algorithm to interpolate the "missing" values so that you have a complete data set with equal time intervals. – Jon Oct 25 '17 at 15:49