Consider that I use k-fold cross validation and select the best model based on the smallest mean error or using some heuristic to choose the best model. After I chose the best model, if I use the full training data to train this model, would it be guaranteed to overcome overfitting (or less overfitting) in deep neural networks?
Is it a good idea to train with the full training dataset after k-fold cross-validation, or again do another set of cross validation (split training dataset to training and validation) and then train the best model on the part of training data?
I read other questions regarding k-fold cross validation but none of them fully convinced me that when selected deep network model trained on whole training dataset will not overfit. As deep neural network, I mean the number of parameters to train P are far more than number of full training samples N (P>>N).