Proper way of using recurrent neural network for time series analysis Recurrent neural networks differ from "regular" ones by the fact that they have a "memory" layer. Due to this layer, recurrent NN's are supposed to be useful in time series modelling. However, I'm not sure I understand correctly how to use them.
Let's say I have the following time series (from left to right): [0, 1, 2, 3, 4, 5, 6, 7], my goal is to predict i-th point using points i-1 and i-2 as an input (for each i>2). In a "regular", non-recurring ANN I would do process the data as follows:

 target| input
      2| 1 0
      3| 2 1
      4| 3 2
      5| 4 3
      6| 5 4
      7| 6 5 


I would then create a net with two input and one output node and train it with the data above.
How does one need to alter this process (if at all) in the case of recurrent networks? 
 A: Another possibility are Historical Consistent Neural Networks (HCNN). This architecture might be more appropriate for the above mentioned setup because they eliminate the often arbitrary distinction between input- and output-variables and instead try to replicate the full underlying dynamics of the whole system via training with all observables.
When I was working for Siemens I published a paper on this architecture in a book by Springer Verlag:
Zimmermann, Grothmann, Tietz, von Jouanne-Diedrich: Market Modeling, Forecasting and Risk Analysis with Historical Consistent Neural Networks.
Just to give an idea about the paradigm here is a short excerpt:

In this article, we present a new type
of recurrent NN, called historical
consistent neural network (HCNN).
HCNNs allow the modeling of
highly-interacting non-linear
dynamical systems across multiple time
scales. HCNNs do not draw any
distinction between inputs and
outputs, but model observables
embedded in the dynamics of a large
state space.

[...]

The RNN is used to model and forecast
an open dynamic system using a
non-linear regression approach. Many
real-world technical and economic
applications must however be seen in
the context of large systems in which
various (non-linear) dynamics interact
with each other in time. Projected on
a model, this means that we do not
differentiate between inputs and
outputs but speak about observables.
Due to the partial observability of
large systems, we need hidden states
to be able to explain the dynamics of
the observables. Observables and
hidden variables should be treated by
the model in the same manner. The term
observables embraces the input and
output variables (i. e. $Y_τ := (y_τ, u_τ)$).
If we are able to implement a
model in which the dynamics of all of
the observables can be described, we
will be in a position to close the
open system.

...and from the conclusion:

The joint modeling of hidden and
observed variables in large recurrent
neural networks provides new prospects
for planning and risk management. The
ensemble approach based on HCNN offers
an alternative approach to forecasting
of future probability distributions.
HCNNs give a perfect description of
the dynamic of the observables in the
past. However, the partial
observability of the world results in
a non-unique reconstruction of the
hidden variables and thus, different
future scenarios. Since the genuine
development of the dynamic is unknown
and all paths have the same
probability, the average of the
ensemble may be regarded as the best
forecast, whereas the bandwidth of the
distribution describes the market
risk. Today, we use HCNN forecasts to
predict prices for energy and precious
metals to optimize the timing of
procurement decisions. Work currently
in progress concerns the analysis of
the properties of the ensemble and the
implementation of these concepts in
practical risk management and ﬁnancial
market applications.

The paper is now finally available in full here: Zimmermann, Grothmann, Tietz, von Jouanne-Diedrich: Market Modeling, Forecasting and Risk Analysis with Historical Consistent Neural Networks.
A: What you describe is in fact a "sliding time window" approach and is different to recurrent networks. You can use this technique with any regression algorithm. There is a huge limitation to this approach: events in the inputs can only be correlatd with other inputs/outputs which lie at most t timesteps apart, where t is the size of the window.
E.g. you can think of a Markov chain of order t. RNNs don't suffer from this in theory, however in practice learning is difficult.
It is best to illustrate an RNN in contrast to a feedfoward network. Consider the (very) simple feedforward network $y = Wx$ where $y$ is the output, $W$ is the weight matrix, and $x$ is the input.
Now, we use a recurrent network. Now we have a sequence of inputs, so we will denote the inputs by $x^{i}$ for the ith input. The corresponding ith output is then calculated via $y^{i} = Wx^i + W_ry^{i-1}$.
Thus, we have another weight matrix $W_r$ which incorporates the output at the previous step linearly into the current output.
This is of course a simple architecture. Most common is an architecture where you have a hidden layer which is recurrently connected to itself. Let $h^i$ denote the hidden layer at timestep i. The formulas are then:
$$h^0 = 0$$
$$h^i = \sigma(W_1x^i + W_rh^{i-1})$$
$$y^i = W_2h^i$$
Where $\sigma$ is a suitable non-linearity/transfer function like the sigmoid. $W_1$ and $W_2$ are the connecting weights between the input and the hidden and the hidden and the output layer. $W_r$ represents the recurrent weights.
Here is a diagram of the structure: 

A: You may also consider simply using a number of transforms of time series for the input data.  Just for one example, the inputs could be:


*

*the most recent interval value
(7) 

*the next most recent interval
    value (6)

*the delta between most
    recent and next most recent (7-6=1)

*the third most recent interval
    value (5)

*the delta between the
    second and third most recent (6-5=1)

*the average of the last three
    intervals ((7+6+5)/3=6)


So, if your inputs to a conventional neural network were these six pieces of transformed data, it would not be a difficult task for an ordinary backpropagation algorithm to learn the pattern.  You would have to code for the transforms that take the raw data and turn it into the above 6 inputs to your neural network, however.
