I have a time series regression prediction problem. So I divided the dataset into 3 parts:
- training (first 70% of the time series data)
- validation (from 70% to 85% of the time series data)
- test set (last 15% of the data)
Then trained the model for some epochs and used
earlystopping callback (
keras callback) on validation set. By using
earlystopping, I will be able to stop the model from further training, if no improvement is detected on the validation set.
Then calculated errors of the predictions on each dataset. Here are the results:
- Training dataset Mean-Squared-Error = 921.4
- Validation dataset Mean-Squared-Error = 1200.2
- Test dataset Mean-Squared-Error = 300
From this question I concluded that my model is acting normally, because Training error is not higher than validation error.
I know that my Test dataset is easier than training and validation sets to predict and this is the reason for lower error. Does my model have problems? Should always test and validation errors be more than train errors? Is my model good at generalization?