From the paper To understand deep learning we need to understand kernel learning, three questions:
In section 2 "Setup" there appears a definition of interpolated classifier as an algo that has zero or close to zero training error and overfitted as an algo "if the same holds for classification loss".
Question 1: what is the mathematical definition of these two types of classifiers? If an interpolated classifier only depends on the training data, then an overfitted algo in this context could mean an algo that has zero or near zero test error (i.e. generalization error)...which seems different than more traditional definitions of "overfit".
Question 2. The authors then go on to state that in their definitions an interpolated classifier is necessarily an overfit classifier -- why?
Question 3. In section 4, does the t-overfits the data definition depend on a fixed training set? I.e. do they mean t-overfit a fixed but arbitrary training dataset?