I am trying to train a neural network on some time series data and decided to implement cross validation for my model.
The cross validation method I'm trying to implement is the Day Forward-Chaining from this post where each validation step uses more training data than the previous step.
1. train: 1, 2 -> val: 3, 4 -> test: 5, 6 2. train: 1, 2, 3, 4 -> val: 5, 6 -> test: 7, 8
As tensorflow requires its samples to be of the same size a sollution is to pad the smaller inputs as mentioned in How to feed input with changing size in Tensorflow.
What I would like to know is whether pre or post padding the sequences makes any difference. Would the model assume these are the same?
x, x, 1, 2 -> here the fist time step is now the third. Will the model be aware of this? 1, 2, x, x -> but here the padding occurs where the validation data should go.
Is there a proper way of doing this or does it not matter?
Another interesting question which I came accross when writing this is Padding - which values for standardized time series data?
What is the best value to use when padding a sequence? My data contains zeros so those should not be ignored. I though of -1 as data is never negative but standardizing it will change that.