I have built an LSTM model for multivariate time series (6 time series). I am predicting one of the variables given the expected values of other five variables. Goal is to compare the accuracy of the LSTM model to Vector autoregression(VAR) model that I built earlier. LSTM gives me better accuracy than VAR. So, should I stop tuning the LSTM any further or should I tune it further for gain in accuracy?
This is a general question which often comes up when tuning deep learning and machine learning algorithms such as recurrent neural network, multilayer perceptron or SVM etc.
When should one decide to stop tuning a neural network any further. I understand that there could be many kinds of constraints such as a deadline when the tuned model is required, etc. for a particular problem but what I am asking is in general. Or one should simply stop tuning a network anymore when one feels that s/he has achieved the desired prediction accuracy or should one try to achieve even better accuracy.
There are plenty of guidelines available on hyper parameter tuning/optimization but its very difficult to find one which summaries when to stop tuning the network any further and stop.