5
$\begingroup$

I have a basic understanding of generative models, like Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs). It seems that they are mostly used to in the field of image processing/computer vision as in https://blog.openai.com/generative-models/

But somehow after a very long quest in searching for ways to emulate/simulate non-stationary time-series processes, i got to generative models in mahcine learning.

Does anyone know of any way that generative models have been used on time series data? especially to generate similar time series out of observed ones?

If anyone can suggest another approach to modelling and simulating time-series data that would also be of great help!

$\endgroup$
3
$\begingroup$

For time series, you want to use LSTM or other recurrent neural network instead of unsupervised generative models in deep learning. You can also convert it to a supervised model which uses $x_t$ to predict $x_{t+1}$ with whichever machine learning algorithm you like.

As you can see in the blog post and in wikipedia of generative model: "In probability and statistics, a generative model is a model for randomly generating observable data values, typically given some hidden parameters." You should note the input to the generative model is some hidden parameters which are usually meaningless, while you want the input to be $x_t$ when generating $x_{t+1}$.

You should also note in the blog post of OpenAI that "Autoregressive models such as PixelRNN instead train a network that models the conditional distribution of every individual pixel given previous pixels (to the left and to the top). This is similar to plugging the pixels of the image into a char-rnn, but the RNNs run both horizontally and vertically over the image instead of just a 1D sequence of characters." So the PixelRNN is basically like a recurrent neural network except that the sequence is on 2D directions while you only have time as your 1D direction. This still just leads you to LSTM.

If you really really really want to use fancy algorithms such as GAN, here is what you can do: forget you have a time series. Usually when time series data are generated, the data are generated at one step after another according to $p(x_{t+1}|x_{1..t})$, which is like PixelRNN. Instead, just think about the data as a bunch of numbers, or a 1D image in analogy to image generation, and now you have a perfect analogy to image generation using GAN: each whole time series is a single training data for you GAN. This method totally ignores some characteristics of time series, for example causality, and just regards your data as a bunch of numbers. I highly doubt this will work well but I encourage you to try.

$\endgroup$
  • $\begingroup$ What to do if we need to look at multiple timesteps before predicting nexttime step. Is LSTM the only option? I am not looking to predict the next time step. Just looking to generate data with multiple time series in it. Will temporal probabilistic graphical models be good? $\endgroup$ – gnjago Oct 18 '17 at 8:02
  • 1
    $\begingroup$ @gnjago Any supervised generative model will do. The key here is to use supervised model instead of unsupervised GAN. You can simply regard whatever you want to generate as y and whatever you want to condition upon as x, no matter if x is single step or multistep. $\endgroup$ – DiveIntoML Oct 19 '17 at 18:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.