I have been thinking about the way of splitting the data into training, validation and test sets for the stateless LSTM. For me, the intuitive way is to arrange the original data into the 3D form (batch_size, time_step, variable_number), then randomly split it into 90%, 5%, and 5%. However, both validation set and test set have a lot of data points overlapping to the ones in the training set even though they belong to different sequences. I guess it is OK for the validation set, but not for the test set because many data points have been used to build the model. If so, should I randomly separate 5% consecutive data points from the original data as the test set before arranging the original data to the 3D form? Note: @Tim♦ The question you demonstrated is more about stateful LSTM and cross-validation. I am asking the questions about the validation for the stateless LSTM.