I was reading "The Elements of Statistical Learning Book by Jerome H. Friedman, Robert Tibshirani, and Trevor Hastie" where I encountered the following:
The part tells us that the RSS criterion will lead us to infinitely many solutions and in order to obtain a unique solution we need to put some restrictions on the RSS criterion.
The book further discusses the following methods to impose such restrictions.
- Roughness penalty
- Kernel methods
- Basis Functions and Dictionary Methods
I very specifically want to understand:
- These methods with some simple examples
- The intuition behind each of these methods
- Please suggest an appropriate source for better understanding