I've a time series from January 2010 to December 2017. I'm using 2010 to 2015 as the training set and I'm choosing the best model trained on the training set based on its performance in the validation/test set which is from January 2017 to December 2018 as measured by the validation/test MAPE.My objective is to have the validation/test MAPE as small as posdible (business requirement) Can the validation and test sets be the same in this case?
Normally, the validation set is used for deciding about the parameter values of the chosen model family, while the test set is used to evaluate the performance of the learned model. The test set should not be used during training at all, while validation set is used during training for choosing the suitable parameter values for the model. In other words, the test set is used only once to report the performance of the learned model, while validation set is used many times to evaluate different parameter settings.