I'm trying to use LSTM neural netwroks for forecasting time series. I am able to generate "OK" looking forecasts, but there seems to be a slight change in the predictions each time I run the model, which I assume is due to the change in the initial weights of the network not being the same each time. See for example below: three forecasts generated using the exact same network.
What is the best way to deal with this if we want to use NNets for time series forecasting?
In other applications, where the data is stable (i.e. stationary) on simply tries over and over until they get the best results on an evaluation set, and then they put that particular model in production.
But in the time series case, you would have to regenerate your forecast (i.e. retrain your network) every time a new value for the time series comes in. How is the sensitivity and instability handled in such a situation?