2
$\begingroup$

I am going through the ISLR book (by Hastie, et al.). In the chapter on model validation, the author suggests that we build the final model on the full data instead of the training data only. The way I always understood the process is that we build the model on training set, validate it with test set and finalize the model. If we use the full data to build the model then what is the point of anything that we did before that step? Thank you!

$\endgroup$
2
$\begingroup$

The point of breaking up the data is to characterize model performance and tune hyper parameters. Building the model on all of the data gives you the largest possible sample size for the purpose of parameter estimation. Two different goals in these phases, so two different procedure.

$\endgroup$
  • $\begingroup$ I like this "Building the model on all of the data gives you the largest possible sample size for the purpose of parameter estimation", but still what if fitting on all the data will give you a weaker model against real-world data? $\endgroup$ – fabiomaia Jan 24 '19 at 15:46
  • $\begingroup$ You train on the training set and validate (during training) on the validation set, which should yield the best model among many. Then you retrain the best model on all available data (training and validation) which should yield a model with different parameters. Isn't it possible that these parameters do not perform so well on real-world data? How do you know that the parameters from training on all data are better than the parameters from training on just the training data? $\endgroup$ – fabiomaia Jan 24 '19 at 20:13
  • $\begingroup$ Thank you for the suggestion. I have done so in stats.stackexchange.com/questions/395645/… $\endgroup$ – fabiomaia Mar 4 '19 at 23:06
  • 1
    $\begingroup$ @fabiomaia Looks like I missed the boat on giving the best answer -- the answer you have from Matthew Drury is the correct one, and if I'd been more on the ball I probably would have given that answer in comments here. Whoops! $\endgroup$ – Sycorax says Reinstate Monica Mar 5 '19 at 0:17

Not the answer you're looking for? Browse other questions tagged or ask your own question.