How to predict with a stateful LSTM the next values I trained a RNN using LSTM cells. I would like to make a predictions for the next 14 days. I get results that are plausible but after reading various blogs I'm not so sure if I'm doing the right thing.
The data I'm using is a time series. This is the shape of my data:
X_train: (1260, 1, 1) 
y_train: (1260, 1)

Where X_train is y_train shifted by -1 (lag 1).
I'm using keras API for setting up my model.
Given is the following model structure.
model = keras.models.Sequential()
model.add(keras.layers.LSTM(units=units_fst_layer, batch_input_shape=(batch_size, moving_window, number_features), 
                            stateful = True, return_sequences=True, dropout=dropout))
model.add(keras.layers.LSTM(units=units_snd_layer, stateful = True, return_sequences=False,
                            dropout=dropout))
model.add(keras.layers.Dense(1))
optimizer = keras.optimizers.Adam(learning_rate=lr)
model.compile(loss='mean_squared_error', optimizer=optimizer)
model.summary()

Model: "sequential_42"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
lstm_83 (LSTM)               (14, 1, 62)               15872     
_________________________________________________________________
lstm_84 (LSTM)               (14, 31)                  11656     
_________________________________________________________________
dense_41 (Dense)             (14, 1)                   32        
=================================================================
Total params: 27,560
Trainable params: 27,560
Non-trainable params: 0
_________________________________________________________________

Then the model gets fitted:
loss = list()
val_loss = list()
for i in range(nb_epochs):
    print(f'Epoch {i + 1}')
    history = model.fit(X_train, y_train,
                        epochs=1, batch_size=batch_size, verbose=1, shuffle=False)
    model.reset_states()

Now comes my question how would I make a prediction for the next 14 days? Which kind of data should I use? The input shape of the array that I have to feed to model.predict() has to have the shape of at least [14, 1, 1]? 14 because this is my defined batch size. This will give me an output array of the shape [14, 1].Is this the prediction of the next 14 days. I guess not right?
Thank you for your answers?
 A: You have to feed in the RNN's own predictions into itself. Why?
From what I know, you can only answer this question going back to the probabilistic interpretation. You basically use an LSTM to model a conditional distribution, although packages like Keras are don't bother telling the user about this. In general, I advice people to familiarize themselves with this. What follows is a brief exposition.
You are using the RNN to model the distribution
$$
p(x_t | x_{1:t-1}),
$$
and you want to use that to get
$$
p(x_{t+k} | x_{1:t-1}),
$$
where $k$ is the prediction horizon. We need some probability theory to get an insight what the latter actually is. Using the sum-rule we get
$$
\begin{align}
p(x_{t+k}~|~x_{1:t-1}) =&~ \int p(x_{t:t+k}~|~x_{1:t-1})~dx_{t:t+k-1} \\
=&~ \int p(x_{t:t+k}~|~x_{1:t-1})~dx_{t:t+k-1} \\
=&~ \mathbb{E}_{x_{t+1:t+k-1}} \left [ p(x_{t+k}~|~x_{1:t+k-1}) \right ].
\end{align}
$$
We an now apply this recursively and will finally arrive at $k=0$ where we can put in our LSTM model.
Depending on your probabilistic assumptions–e.g. if you are using a squared loss you are implicitly using a Gaussian distribution–you need to sample from your LSTM's output in different ways. You can also try feeding in the point predictions (i.e. not sampling), but that answer will only be approximate and underestimate uncertainty.
A: Your case, in which you explicitly feed the network with the previous value, is a little bit different than the usual case, i'll explain about the general case later. First to your case: Let's ignore for a moment the batch size used for training. After training a model, you can predict with batch size of 1 i.e. the next value. If you want to predict the value of $y_t$ you need to feed your LSTM with the features $x_t$, in your case $x_t = y_{t-1}$. So you can imagine having a loop, each time you save the previous prediction and then feed it to the LSTM (along with other features if you have them). If you want prediction for 14 days, you will have to run it day over day.
Now regarding batch size. First you need to distinguish between 2 parameters, batch_size and num_steps which is how many time steps you train together in order to predict the next value. What you need, in your case, is batch_size = 1 & num_steps = 1.
In the general case, LSTM will feed the last value $y_{t-1}$ for you automatically. This is how it "remembers" the past (or more accurately it will feed the last hidden state, $H_{t-1}$, which represents the previous values $y_{t-1}$ (and recursively also older $t$'s).
For example: Time series in which the features are the daily revenue of a company $C$ i.e. $x_t$ = the revenue of company $C$ at day $t$ and the label is the stock price. i.e. $y_t$ = the stock price of company $C$ at day $t$.
In this case you can feed the LSTM with batch of $x_t$ and get the corresponding $y_t$ predictions.
