What is the difference between training, test, and holdout sets?
I know these concepts, just want to ensure that I have understood correctly.
Training set is something that we have as of now. We will remove subset from it and removed subset will be called holdout set.
We will build models using remaining data (what remains after removing holdout set) and the holdout set is used to finalized estimates of tuning parameters (step 1)
Then we will build a final model on the entire Training set (including holdout set). Tuning parameters values are same as that we got from step 1.
Test data is something that we get in future. We don't know their Y/dependent variable value and we predict it using our model.