Continue the predictions beyond the current data using time series neural network

Question

I have a single time series variable and I want to train a neural network in a sort of auto-regressive fashion. specifically, I have data for water consumption that is changing with time

In the above figure the x-axis in the figure represent the number of month. I divided data into training data and validation data. But my problem is I want to continue the predictions beyond the current data not just up to the validation data. I know there are many program can apply neural network time series analysis like matlab or r or encog library on java but no of them as I know can predict beyond the current data what you recommend guys?

Answer 1

What you are looking for is a recurrent neural network, and in particular the current favorite flavor, the LSTM. The amount of data you have is miniscule, but since the pattern is simple you may get it to work with a tiny network that has a handful of neurons.

There are a number of state-of-the-art neural network toolboxes, although they are still maturing and only partially plug-and-play. Worse, they may not offer out-of-the-box support for LSTMs. There is Caffe, Torch, Theano, Nervana Systems Neon, etc. They work mostly on Python and Linux.

However, I can recommend the perfect starting point. Andrej Karpathy did a fun piece of research that garnered a lot of press in which he trained an LSTM network on text corpuses (such as Shakespeare and the Linux source code) and used it to generate more text. The results, to put it objectively, were not as awful as one might expect. Anyway, his code, which uses Torch, is simple, well-commented, and available on GitHub. You should read his detailed blog post as well.

I think you can repurpose his code for your problem quite easily. You can quantize and encode the numerical values as ascii characters, and use the char-rnn scripts to train on the data as well as produce a prediction. Hint: when using sample.lua for prediction, set the argument -primetext to the last data in the time series as the starting point for prediction, and set the argument -temperature low to generate the most expected progression. Have fun!

Answer 2

I think that you can clearly see the pattern and seasonality (probably the peak some where in the summer) in this time series and you don't need to train complex network for this issue. You can just model it with function. For example you can choose some polynomial (or better some other function which can monotonically decrease) which will fit it in a line and add some sin or cos function to control the time period (or maybe 2 additional terms). Don't use complex tools for simple tasks.