In Learning from Data course taught by Caltech Professor Yaser Abu-Mostafa the following notation is used to describe the in sample and out of sample errors.
where $h(x_n)$ is our selected model, and $f(x_n)$ is the target function.
It's clear that in sample error is only an empirical estimate of the out of sample error.
My question is the following, in order to evaluate our model we use the testing accuracy instead of the training accuracy. Assuming that the samples are i.i.d, is there a mathematical reason using testing accuracy is a better evaluation of the out of sample performance of the model than the training accuracy?
I'm aware of the notion of overfitting. However, I'm interested in a mathematical description which leads us to the motivation of separating training and testing sets.