I have time series data and am fitting a (LSTM) neural network. The time series data include let's say a brain wave (var1) as well as the previous state (prev_state) and I want to predict a state (y_pred) based on the data and values over several steps before. The problem I run into is I can fit a neural network, or any type of model, on the training data but the training data already has all the correct state values; but when predicting one record at a time over the same training data everything gets thrown off and the model can get stuck in one state because a state prediction is used as input to the next record.
Training Data: Var1 prev_state y_true 22.2 1 0 20.1 0 1 25.1 1 1 28.5 1 1 30.0 1 1
Predicting not one a time but with already states known: Var1 prev_state y_pred 22.2 1 0 20.1 0 1 25.1 1 0 (HERE THE MODEL MADE THE WRONG PREDICTION BUT IT'S OK BECAUSE THE TRAINING DATA ALREADY KNOWS THE TRUE STATE IN THE NEXT RECORD) 28.5 1 1 30.0 1 1
Predicting one at a time and then inserting the prediction in the input of the next record: Var1 prev_state y_pred 22.2 1 0 20.1 0 1 25.1 1 0 (HERE THE MODEL MADE THE WRONG PREDICTION AND IN THE NEXT RECORDS ENTIRE PREDICTION SEQUENCE GETS THROWN OFF) 28.5 0 0 30.0 0 0
Is this a problem with over or under fitting? Not sure how to approach this. I'm wondering if the setup is just plain wrong and I'm wasting my time.