# LSTM mimicking unseen time series data during testing

I have built a LSTM network which has been trained on a time series dataset (which is week-wise logged). The LSTM is able to make pretty accurate predictions as of now.

Training data seems to have this trend :

Note that the training data for each of the 3 weeks is almost similar in shape.

LSTM predictions on unseen testing data (week 4 data):

As you can see, the LSTM is still able to trace the unexpected peak in the graph which was never seen in the training data.

Any reason as to how the LSTM is able to do the above?

The requirement is that the model should not be able to trace the unexpected peak and the unexpected peak gets flagged as an anomaly when an output layer on top of the LSTM compares the actual and the model predicted values and detects the large variation.

Is LSTM not a suitable fit for the above scenario?

• Probably you are predicting only one step ahead at a time. In this case a simple x(t) = x(t-1) predictor will also give you a very nice graph. Try it and you will see it. It is a visual thing, like plotting one time step shifted version of a graph on top of the original graph gives the illusion of great prediction. That's why it is a bad idea to assess fit visually. – Cagdas Ozgenc Jun 15 '16 at 5:10
• What means would you suggest to assess fit? – Ashwin Naresh Jun 15 '16 at 5:26
• In prediction business there is no way to measure absolute performance only a relative one. I recommend you compare your error metric, i.e. mean squared error, of your model with the dummy x(t) = x(t-1) model on the unseen test data (not training data). If you have better performance then it means your model gained some information. But it will not mean that you achieved the best model. – Cagdas Ozgenc Jun 15 '16 at 5:35
• Alright, thank you. I will try fitting the data with a GARCH(1,1) model and see how it will work out. – Ashwin Naresh Jun 15 '16 at 6:31
• @AshwinNaresh I have the same problem when I'm using LSTM. Would you please let me know what you decided to do finally and were there improvements over GARCH(1,1) when you tried it? Thanks – ahajib Jul 11 '16 at 16:57

Probably you are predicting only one step ahead at a time. In this case a simple $x(t) = x(t-1)$ predictor will also give you a very nice graph. Try it and you will see it. It is a visual thing, like plotting one time step shifted version of a graph on top of the original graph gives the illusion of great prediction. That's why it is a bad idea to assess fit visually.
In prediction business there is no way to measure absolute performance only a relative one. I recommend you compare your error metric, i.e. mean squared error, of your model with the dummy $x(t) = x(t-1)$ model on the unseen test data (not training data). If you have better performance then it means your model gained some information. But it will not mean that you achieved the best model