I read everywhere that the ideal way of training a model would be to e.g.: run k-fold learning for hyperparameter optimization on 80-90% of the dataset, then test the best model on the rest. As far as I understand, in each iteration of the k-fold, we receive an estimation of the generalization capabilities of our model. Now, given that the dataset is sufficiently large, this performance estimate will be the same as if the same model is run on the test set, is it not? Focusing on a single fold, we have a split of e.g.: 60-20-20 of train/validation/test sets, where the 20-20 splits are both unused for training and are from the same distribution, given a proper random sampling. So then, if we optimize our hyperparameters for the validation split of 20, it should largely be equivalent to optimizing it for the test split of 20. In the end, we want the best generalizing model, so why bother with the test set at all? Using the test set as the final evaluation of our model's performance will be the same biased estimate as was with the validation set either way. We would need a different dataset altogether from a different source to properly evaluate the true, unbiased generalization of our model, wouldn't we?
In short, why not just run hyperparameter optimization with k-folds on the entire dataset, then pick the best model and publish its k-fold averaged performance with disclaimers that it was not tested on different sets? It sounds more correct to me, than pretending that the test split from the same dataset is a better estimator.