Time series forecasting with RNN(stateful LSTM) produces constant values

I have a time series daily data for about 6 years(1.8k data points). I am trying to forecast the next t+30 values, Train data independent matrix (X)=Sequences of previous 30 day values Train (Y)=The 31st day value for each of previous 30 day values.

I followed the following methodology to forecast: Y for t+1 is first forecasted, Then X matrix row is shifted by 1 day and the forecasted Y is appended to the end of this row, then use this row to predict t+2 value and continue. However in each sequence after (usually) t+3 days the forecasted values become constant for the rest of the t+n days.

Is this the correct way to forecast time series with LSTMs?

How can this behaviour be explained?

Should I rather try to train the network with Y matrix having 31st day to 60th day values for the same X?

My train data looks something like this:

array([[-0.35811423, -0.22393472, -0.39437897, ..., -0.36718042, -0.37080689, -0.35267452], [-0.22393472, -0.39437897, -0.13327289, ..., -0.37080689, -0.35267452, -0.2030825 ], [-0.39437897, -0.13327289, -0.1532185 , ..., -0.35267452, -0.2030825 , -0.25294651],

Architecture: Input LSTM layer (20 neurons) 1 hidden lstm layer 20 neurons 1 output dense layer, batch size as 1. Stateful lstm, I reset model states after each epoch.

• I suggest you narrow down your list of questions. Here is some helpful advice on how to ask a good question: stats.meta.stackexchange.com/a/1483/121522 – mkt Aug 7 '17 at 21:36
• What loss function are you using for your network? I'd recommend L2/MSE to start with for all regression problems unless you have a specific loss function that is specifically applicable. If you can, publish some data points on loss metrics at specific epochs (1st epoch, 10th, 100th, etc.). It may help to also go back and predict against the training set so you can plot the back-fit (model vs. actual). Batch size of 1 is not typically recommended, but it should still work (you can try 128 or some such). Are you using bias weights in your network? This may be required to make it work. – T3am5hark Apr 6 '18 at 20:41
• I am using mape as error metric. What pitfalls can this have versus mse? What is the reason for not recommending batch size 1 ? Changing this didnt help. Plotting the data at different epochs: do you mean to do this in the context of dying reLu? ( it could go to zero after few epochs?) Any other intuition behind this? – Narahari B M Apr 8 '18 at 15:14