In "Introduction to Statistical Learning", for maximal margin classifiers, they say:

"Although the maximal margin classifier is often successful, it can also lead to overfitting when $p$ (the number of dimensions) is large," (p. 341)

also in "Elements of Statistical Learning":

"Again one can enlarge the space using basis transformations, but this can lead to artificial separation through over-fitting" (p. 135)

I don't seem to understand this. Could someone please explain why a large dimension can cause the separating hyperplane to overfit? How does increasing the dimension makes separation possible? (even if it's just artificial)?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.