I'm new to neural networks and machine learning and I was wondering how you use time series data to set the weights of a regular FNN, and how you use the ending weights to forecast the time series. In essence how can you transform the time series data into weights and back again for the output.
A feed-forward neural network (typically multi-layer) is a type of supervised learner that will adjust the network weights on its input and internal nodes, in an iterative manner, in order to minimize errors between predicted and actual target variables. It commonly uses stochastic gradient descent (sometimes called error back propagation) over many iterations in order to find a local minimum of the error response and optimize the network weights accordingly.
The basic idea behind stochastic gradient descent is to start by randomizing the weights, then adjust them by iterating through several passes and updating the weights in a direction that moves the total error between target and predicted errors towards the local minimum error of the gradient surface. In practice, a tradeoff is found between optimizing a training set against a validation set, in order to reduce the problem of over-fitting.
Lastly, the input (time series or otherwise) often needs to be transformed in order to create a stationary series that is also bounded (amplitude wise) between the input range of the NN layer transfer function(s)(typically, 0 to 1 or -1 to 1).
Once the weights have been trained, the model can be stored and used to process additional new time series data, much like a typical linear regression based model.
An example illustration of using a NN to predict finanacial time series data, using Weka, is posted here: http://intelligenttradingtech.blogspot.com/2010/01/systems.html
A good text comparing financial AR based models against NN models is, "Applied Quantitative Methods for Trading and Investment," Christian Dunis et.al.
Let me start by answering your question, then I will add some comments and suggestions.
First, I believe that when you say "weight" you actual mean "input"/ "output". This is because you asked how to transform the time series to weights and how to transform output weights into a prediction. Neural network terminology uses "weight" to mean something else (pat's answer uses the term "weight" correctly).
This is what people usually suggest: If your time series looks like
X_1, X_2, ..., X_n, ...
then you do the following:
Step 1: Decide how many observations you want to use to make a prediction.
Step 2: Decide how many steps forward you want to predict.
Both these choices are fixed for the NN.
For this example let's say you want to use the last 5 readings to make 2 predictions.
Then you will
Step 3. Create a neural network with 5 input nodes and 2 output nodes.
Step 4. Create your training set with each element consisting of 5 sequential readings as input and the next two readings as output.
Here are the first TWO elements of the training set:
Input = X_1, X_2, X_3, X_4, X_5 Output = X_6, X_7 Input = X_2, X_3, X_4, X_5, X_6 Output = X_7, X_8 etc.
Hopefully that answers your question.
Now for some gyan.
If your data is noisy e.g. stock ticks, then my feeling is that this will be hard to train. I know I have had bad luck trying to train neural networks on noisy data.
So here is another strategy:
First model your time series using the ARIMA framework. This views a time series as
Polynomial base + Cyclic component + Bounded randomness
(Take a look at the Weka example in pat's answer from this point of view.)
Now my feeling (and I am still experimenting) is that the random component(s) are interfering with the training of the NN. So I want to avoid trying to predict them directly.
Picture your series data coming in. On every reading you feed it into your ARIMA black box, which figures out the underlying model and then spits out the ARIMA model parameters. So at time 0 you have a set of parameters, and then at time 1 you have an updated set of parameters, etc.
Note: The ARIMA black box is slow.
Question 1: Would it be possible for a neural network to learn how these parameters change? My feeling is that they will change slowly, so this may be doable.
Question 2: Could you train a different neural network to discern any patterns in the ARIMA error? I.e. If ARIMA predicts 5.4 and the actual next reading is 5.5, could you train a neural network to figure out that 0.1?
If you have access to a MATLAB installation, try the neural network toolbox first: http://www.mathworks.com/help/toolbox/nnet/gs/f9-56659.html (have a look at the screenshots)
It is very, very good for what you're trying to accomplish, and with great documentation. It is a great starting point.
There are several ways in which you can 'train' a neural network. Personally, I prefer the genetic algorithm approach - each individual represents a set of weights, with the fitness function being the performance of the neural network.
The performance of the neural network in terms of time series analysis could be the mean squared error of the predictions against the targets. One common method in time series prediction with neural networks is to use the % change from specific intervals over a 'lookback' period.
You may find this useful - http://ijcai.org/Past%20Proceedings/IJCAI-89-VOL1/PDF/122.pdf