How to train a Recurrent Neural Network model for time series with bursty events

Question

I am trying to predict based on a time series of the inputs, using a Neural Network. I am currently considering an RNN or LSTM architecture, however, I am open to any reasonable suggestion.

The problem is that the input data may be busty (e.g. I may have a lot of events for some minutes, and none or very few at other minutes...) Another problem is I really do not care about my prediction at all times, just at say every minute.

I could create a network that would take events at the maximum frequency, and introduce non-event events for all times there are no real events. However,I feel that this would be very inefficient, as the network would have to process all these fake events, plus the resulting network would be more difficult to train and much easier to over-fit...

I could just ignore the time aspect, and introduce events as they come, introducing some events that signal a time interval... The problem if i do this however, is that when fitting the network, times with lots of events would weight more (could I specify that outputs at certain times do not matter?)

[Edit:] I could also encode time as another feature of delta-time to the previous event, but this probably downgrades the significance of event order, plus there is the problem of how to create (or consider for testing) outputs only at specific times...

Is there any approach that has been tried or sounds good or any relevant documents or tutorials on how to approach this problem?

Answer 1

All of the approaches you have mentioned are valid ones: I would go with the last one (timedelta) - in my experience it produces good results. If you don't want to weight some of the intermediate outputs you could use a temporal sample weights from keras and set their weights to almost 0 effectively excluding their contribution to the average loss.