How to design RNN\VAE for sparse sequential data?

Question

I am currently struggling with designing some Deep Learning algorithms that takes as an input sparse sequential data, since the system I try to model signals sparsely (once in a while). I try to predict the next signal (strength, timing, etc). Can you suggest an architecture\design\method\resources for this type of problem?

Answer 1

Have you looked at this paper? A Recurrent Latent Variable Model for Sequential Data (https://arxiv.org/abs/1506.02216)

For the sparsity, a few works I have known use time index as an additional input to the RNN (such as Recurrent Recommenders Networks (http://alexbeutel.com/papers/rrn_wsdm2017.pdf)).

Not sure if these suggestions will work for you, but at least you can try.