# Choosing a Generative Models for time series data

I'm looking to try and generate economic time series data (GDP, Inflation, Unemployment etc.) with some generative models. I am thinking of using them with LSTMs as well as more AR(p) style models so I would prefer models that generate long strings of data (ie inflation from t = 0 to t = 20 or something) but if that doesn't generate good results, which I'm guessing it might not, I would also be fine with an AR(p) style results (ie output is only t = 0 and t = 1).

I'm considering using both Generative Adversarial Networks (GANs) or Deep Belief Nets (DBNs) to generate my data, but I'm unsure which I should focus my research efforts into. One thing I don't need is deep convolutional layers for image generation that GANs are good at, and a model overfitting the time series is also a problem I want to avoid (economic time series have like maybe 5000 observations accross 50 countries, so the GANs would probably have to be pretty shallow). On the other hand, I have read that Restricted Boltzmann Machines are considered toy models now, (unsure if DBNs are also considered that). Also unsure of performance of DBN with Real Valued Output.

If you have other suggestions on other types of models I could use, definitely let me know. However, I really want as state of the art from a Machine Learning perspective as possible (considering I'm a single person with good machine learning and neural network experience, but not a professor with a lab and 30 years of experience studying generative models). I also understand that economist don't typically generate data to improve model performance, but this is a research project and I want to see if I can improve model performance (and hopefully understanding causality) better with some generated data.

I understand that this question is perhaps subjective, but perhaps if someone could explain the pros and cons of both DBNs and GANs. And how helpful each would be for my problem.

• Did you get anywhere with this? Mar 13, 2017 at 12:43
• Nope not yet, but mainly because I was doing other stuff. After more research it seems like GANs are the way to go.
– www3
Mar 14, 2017 at 21:10
• Similar question on modeling time series using generative models is available on reddit. Jun 12, 2017 at 15:31

What you are considering is to be able to sample from $p(x_{t+1}|x_{1...t})$, which should be captured by LSTM or AR models you used. The key here is that the probability is a conditional probability, while the generative models are unconditional probability $p(x)$. For example, most generative model takes as input some relatively meaningless latent variable value, so you cannot even make a model that takes $x_{1..t}$ as your input.
What I would suggest is to simply assume an AR(1) type process, and takes $x_t$ as input to a simple multilayer perceptron to predict $x_{t+1}$. This is kind of like a model with complexity in between your AR linear model and LSTM, but the multilayer perceptron can capture more nonlinear dependency than AR linear model and has less variance and is easier to train than LSTM.