Ian Goodfellow in his book writes that When we use kernel trick to get an infinite-dimensional vector, we can always have enough capacity to fit the training set, but generalization to the test set often remains poor. Why does the generalization remain poor?
$\begingroup$
$\endgroup$
1
-
2$\begingroup$ Hi: there might be special details because it's a kernel but, simply speaking it's due to over-fitting just like one can over-fit when building any model in machine learning-statistics. $\endgroup$– mloftonAug 11, 2019 at 8:22
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
Goodfellow's example seems like a specific example of the bias-variance tradeoff. Fitting training data is pretty easy with modern machine learning methods; generalizing to unseen data is much harder. Striking the balance is where most of the work happens.
We have some threads about overfitting which might help.
answered Aug 13, 2019 at 19:41