4
$\begingroup$

I have built a LSTM network which has been trained on a time series dataset (which is week-wise logged). The LSTM is able to make pretty accurate predictions as of now.

Training data seems to have this trend : Training Data

Note that the training data for each of the 3 weeks is almost similar in shape.

LSTM predictions on unseen testing data (week 4 data): LSTM predictions

As you can see, the LSTM is still able to trace the unexpected peak in the graph which was never seen in the training data.

Any reason as to how the LSTM is able to do the above?

The requirement is that the model should not be able to trace the unexpected peak and the unexpected peak gets flagged as an anomaly when an output layer on top of the LSTM compares the actual and the model predicted values and detects the large variation.

Is LSTM not a suitable fit for the above scenario?

Thanks in advance.

$\endgroup$
  • 4
    $\begingroup$ Probably you are predicting only one step ahead at a time. In this case a simple x(t) = x(t-1) predictor will also give you a very nice graph. Try it and you will see it. It is a visual thing, like plotting one time step shifted version of a graph on top of the original graph gives the illusion of great prediction. That's why it is a bad idea to assess fit visually. $\endgroup$ – Cowboy Trader Jun 15 '16 at 5:10
  • $\begingroup$ What means would you suggest to assess fit? $\endgroup$ – Ashwin Naresh Jun 15 '16 at 5:26
  • 1
    $\begingroup$ In prediction business there is no way to measure absolute performance only a relative one. I recommend you compare your error metric, i.e. mean squared error, of your model with the dummy x(t) = x(t-1) model on the unseen test data (not training data). If you have better performance then it means your model gained some information. But it will not mean that you achieved the best model. $\endgroup$ – Cowboy Trader Jun 15 '16 at 5:35
  • $\begingroup$ Alright, thank you. I will try fitting the data with a GARCH(1,1) model and see how it will work out. $\endgroup$ – Ashwin Naresh Jun 15 '16 at 6:31
  • $\begingroup$ @AshwinNaresh I have the same problem when I'm using LSTM. Would you please let me know what you decided to do finally and were there improvements over GARCH(1,1) when you tried it? Thanks $\endgroup$ – ahajib Jul 11 '16 at 16:57
2
$\begingroup$

In comments, Cagdas Ozgenc writes

Probably you are predicting only one step ahead at a time. In this case a simple $x(t) = x(t-1)$ predictor will also give you a very nice graph. Try it and you will see it. It is a visual thing, like plotting one time step shifted version of a graph on top of the original graph gives the illusion of great prediction. That's why it is a bad idea to assess fit visually.

In prediction business there is no way to measure absolute performance only a relative one. I recommend you compare your error metric, i.e. mean squared error, of your model with the dummy $x(t) = x(t-1)$ model on the unseen test data (not training data). If you have better performance then it means your model gained some information. But it will not mean that you achieved the best model

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.