6
$\begingroup$

I'm looking to try and generate economic time series data (GDP, Inflation, Unemployment etc.) with some generative models. I am thinking of using them with LSTMs as well as more AR(p) style models so I would prefer models that generate long strings of data (ie inflation from t = 0 to t = 20 or something) but if that doesn't generate good results, which I'm guessing it might not, I would also be fine with an AR(p) style results (ie output is only t = 0 and t = 1).

I'm considering using both Generative Adversarial Networks (GANs) or Deep Belief Nets (DBNs) to generate my data, but I'm unsure which I should focus my research efforts into. One thing I don't need is deep convolutional layers for image generation that GANs are good at, and a model overfitting the time series is also a problem I want to avoid (economic time series have like maybe 5000 observations accross 50 countries, so the GANs would probably have to be pretty shallow). On the other hand, I have read that Restricted Boltzmann Machines are considered toy models now, (unsure if DBNs are also considered that). Also unsure of performance of DBN with Real Valued Output.

If you have other suggestions on other types of models I could use, definitely let me know. However, I really want as state of the art from a Machine Learning perspective as possible (considering I'm a single person with good machine learning and neural network experience, but not a professor with a lab and 30 years of experience studying generative models). I also understand that economist don't typically generate data to improve model performance, but this is a research project and I want to see if I can improve model performance (and hopefully understanding causality) better with some generated data.

I understand that this question is perhaps subjective, but perhaps if someone could explain the pros and cons of both DBNs and GANs. And how helpful each would be for my problem.

$\endgroup$
  • 1
    $\begingroup$ Did you get anywhere with this? $\endgroup$ – RockJake28 Mar 13 '17 at 12:43
  • $\begingroup$ Nope not yet, but mainly because I was doing other stuff. After more research it seems like GANs are the way to go. $\endgroup$ – www3 Mar 14 '17 at 21:10
  • 1
    $\begingroup$ Similar question on modeling time series using generative models is available on reddit. $\endgroup$ – Ravi Kiran Jun 12 '17 at 15:31
2
$\begingroup$

I think using generative model from deep learning will not lead you anywhere with the time series economic data you are considering, and GAN will probably be the worst generative model for your application.

What you are considering is to be able to sample from $p(x_{t+1}|x_{1...t})$, which should be captured by LSTM or AR models you used. The key here is that the probability is a conditional probability, while the generative models are unconditional probability $p(x)$. For example, most generative model takes as input some relatively meaningless latent variable value, so you cannot even make a model that takes $x_{1..t}$ as your input.

What I would suggest is to simply assume an AR(1) type process, and takes $x_t$ as input to a simple multilayer perceptron to predict $x_{t+1}$. This is kind of like a model with complexity in between your AR linear model and LSTM, but the multilayer perceptron can capture more nonlinear dependency than AR linear model and has less variance and is easier to train than LSTM.

You mentioned you failed in using AR and LSTM model. Using LSTM is usually hard with high variance data for economic data and can quickly overfit into some unknown region. If you still want to try it, one suggestion is to assume a normal distribution of all your variables, and uses LSTM to output the mean and variance of model prediction.

$\endgroup$
  • $\begingroup$ If there is volatility why not suggest a GARCH or ARCH model? $\endgroup$ – Michael R. Chernick Aug 18 '17 at 2:11
  • $\begingroup$ Sure that is a possibility, I am just responding to the possible choice of deep learning models. You can use whatever traditional model you see fit. $\endgroup$ – DiveIntoML Aug 18 '17 at 3:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.