Time series with neural networks I am new to neural networks (NN), so far I have been playing with the International Airline Passengers time-series dataset. To feed the NN, I transformed the data from:
ti
ti+1
ti+2
ti+3
...
to look like:
ti   ti+1
ti+1 ti+2
ti+2 ti+3
...
and then feed the NN with the latter.
Now I would like to train neural networks on another dataset. I have three different features. The dataset looks like:
ti            si           ri
ti+1            si+1           ri+1
ti+2            si+2           ri+2
ti+3            si+3           ri+3
How can transform the dataset (as done previously) to feed it as input to the neural network?
Would it be something like:
ti            si           ri ti+1            si+1           ri+1
ti+1            si+1           ri+1 ti+2            si+2           ri+2
ti+2            si+2           ri+2
ti+3            si+3           ri+3
...
?
 A: Say you have variables $X_1(t), X_2(t), X_3(t)$, measured at each time point $t$. If using a feedforward network, the input at each time step could be any array containing the values of the variables at the current and previous time steps (over some fixed interval). For example, the vector:
$$[X_1(t), X_2(t), X_3(t), X_1(t-1), X_2(t-1), X_3(t-1), \dots]$$
Or, it could be a matrix:
$$\left [
\begin{array}{ccc}
X_1(t) & X_1(t-1) & \dots \\
X_2(t) & X_2(t-1) & \dots \\
X_3(t) & X_3(t-1) & \dots \\
\end{array}
\right ]$$
The arrangement of the variables in the array doesn't matter if you're using a vanilla, fully connected network; the training procedure will learn the proper weights. But, the arrangement can matter if you're using a network architecture that makes assumptions about how neighboring values in the input are related to each other (e.g. convolutional nets).
If using a recurrent neural network (often used to process time series), the input at time $t$ would simply be the vector $[X_1(t), X_2(t), X_3(t)]$, and a new input would be fed in at each time step.
