What is the procedure of Training, Testing and Validation Test? Explain it thoroughly. Or give some link for related articles


Training: this is the data you use to estimate the parameters of the model. You also obtain in-sample fit diagnostics.

Testing: this is the set which you did not use for estimation. You predict the outcomes of this data set, and obtain out-of-sample diagnostics such as RMSFE (root mean square forecast error).

You generally use training set to estimate and pre-select promising model or filter out bad models. For instance, you look at t-statistics, significance etc to remove hopeless models from consideration.

You use testing set to select the final model, the best one of all. Of course, you take into account the in-sample diagnostics, but out-of-sample metrics would somewhat more prominent.

Validation is used to validate the final model specification. You don't use this set to select the models though! You only use to validate the final model specification. The existence of the validation set is a work around for a fundamental issue with out-of-sample testing, where its use is not much different than in-sample during model selection. So, you have this one final chance to kill the model with the validation set.


Not the answer you're looking for? Browse other questions tagged or ask your own question.