All mainstream machine learning approaches I've seen depend on a test and usually a validation dataset to measure model accuracy during and after training.
This seems like it uses up quite a lot of data that could be used for training.
Are there any machine learning approaches that can estimate model accuracy without extra datasets? I've seen theoretical approaches like VC dimension and Occam learning, which can estimate model accuracy without the extra datasets, but I'm not aware of mainstream, practical applications of these theories.
Also, is there a reason why machine learning engineers prefer to split their datasets instead of using all the data for training and verifying model accuracy in another way?