Some additions to @civilstat's answer:
Now that I have selected near-optimal hyperparameters, I can re-train the model on my whole dataset. There is no validation set though,
... which is why an additional data set to verify the final model's performance is needed whenever you need an estimate of the generalization performance of the final model.
Enter
nested cross validation
an additional held-out set independent of all data used for training the final model, including the test sets used for optimization.
Or, even better, a dedicated validation study for the model/method.
measuring performance during actual prediction use (this may be an ongoing QC to detect when the model becomes unsuitable e.g. due to drift)
In your particular example, random forest provides you automatically with out-of-bag estimates.
(I know that the typical RF implementations do not offer grouped splitting, so the oob-estimate may be optimistically biased. But that affects its training as well...)
I have to wait for a real data point coming from the wild and be confident my model is the best possible.
While optimal and best possible [predictive performance] sound well, in practice, you typically need a model that predicts sufficiently well. Among those, robustness/ruggedness may be more important than the last little bit of performance on data obtained at the same time as the training data.
You judge your final model to be sufficiently good by verifying its performance with a suitable and independent data set.
Could it not be that I have overfitted when training on the whole dataset,
Of course. However, (repeated) k-fold CV can give you additional insight/warnings: if the (i⋅) k surrogate models differ a lot (may be difficult to judge for RF) or their predictions are unstable (differ a lot for the same input case; you can directly measure this for repeated k-fold CV), then your model training is not stable and you are overfitting.
and would not have been better to retain one (assuming there was a way to select) of the k surrogate models
Due to more training data, you'd expect the model trained on the whole data set to be equivalent or even better (slight pessimistic bias of CV) than any single of the $k$ (or $i \cdot k$) surrogate models. So on average, you'll do better to take the model trained on the whole data set.
Side note: which of the surrogate models do you keep? The first, the last, one randomly chosen? Why not another one?
My answer here would be: if model training and optimization went well, they are equivalent, so it doesn't matter. If it does matter, better go for more rigorous training to stabilize the model(s).
If you want to use the performance estimate obtained during cross validation as generalization error, you can not select based on these performance estimates*. Instead you take a pre-determined or randomly (by random number, as opposed to arbitrary picking!) chosen model.
Still, one may argue that a bird in the hand is worth two in the bush: for the surrogate models, you have actual, direct performance estimates. For the model trained on the whole data set, you haven't (except that for RF, you have
the oob-estimate).
Taking one of the surrogate models and using it as final model boils down to doing hold-out rather than cross validation. There are some situations where this may be preferrable.
But consider: the performance estimate for each of the surrogate models comprises only $\frac{n}{k}$ tested cases. This estimate has variance (among other variance components) due to the finite number of tested cases. This variance will be $k$ times as much as the corresponding variance component for the model trained on the whole data set, where predictions from all $n$ cases are pooled into the CV performance estimate. OTOH, for the CV estimate applied to the model trained on the whole data, we expect a slight bias (due to larger training data size and learning curve) and an additional variance component due to model training instability, which we may approximate by the instability observed between the surrogate models.
if training instability dominates, you'd usually want to go back and develop a more stable model. However, if you need to do with what you have, your best bet would still not be taking one of the surrogate models but instead stabilizing the prediction by taking all of them into an ensemble. That is possible with cross validation just like with bootstrapping.
if the variance component due to number of tested cases dominates and training is stable, the model trained on the whole data set is still a better bet than any single surrogate model.
if training is stable, and sample size large, you may go with any (pre-determined/randomly chosen, no arbitrary picking) single of the surrogate models and its performance estimate and skip training on the whole data set.
* Why not select and use error estimate as generalization error
Due to the selection, the error estimate will become biased in a optimistic fashion:
When measuring prediction or generalization error, we observe and summarize the difference between the (surrogate) model's predictions and the true/reference value for the tested cases. Like any other measurement this is itself subject to error, in particular to variance (random error) due to the tested cases. (And the fewer tested cases, the more variance uncertainty, as usual.)
Now keeping aside the actual performance, we're more likely to select a model for which accidentally good performance was observed than a model for which accidentally bad performance was observed. Thus, the selection is biased towards models that happened to look better than they actually are. We are also more likely to select actually better models (which is what we want) but the extent of the selection bias depends on the size of the actual effect (better model) compared to the random error (variance, noise). Bias is expected to be higher for small numbers of tested cases, and for selecting from many models (multiple comparisons).
For the surrogate models of k-fold cross validation we actually have the assumption that they are equivalent, i.e. that we expect them to have (approximately) equal predictive performance. Thus, if we observe differences in the per-fold performance, we should by default expect them to be accidental. (With repeated k-fold cross validation, we can try to separate (in)stability-related variance from sample-size related variance more easily than with a single run.)
