# Link between norm of weights/coefficients and smoothness

We often avoid overfitting by penalizing the norm of the weights/coefficients (in a classic Ridge or Lasso regression). I understand that we want smooth functions as they will be more likely to generalize. But what is the link between smoothness and norm?

• I wanted to write weight's norm obviously. – HumanLearner Apr 8 at 14:14
• I don't understand what you mean by "link", are you looking for a theoretical explanation of penalized regression or something more practical? – RScrlli Apr 8 at 19:32
• Thank you for you answer. I am looking for a theoretical explanation. Why penalizing high coefficients leads to smoother functions ? – HumanLearner Apr 9 at 8:30
• As I see it, the penalisation force the coefficients to shrink, in the case of Lasso actually some coefficients become zero. The smoothness is a consequence of shrinking the coefficients toward zero. – RScrlli Apr 9 at 8:49
• For instance LASSO can be used as a variable selection model, and it's obvious that by reducing the complexity of your model i.e. by reducing the number of covariates the over-fitting will be reduced and hence the approximation will be smoother. (The smoothest function you could imagine is a straight horizontal line where you include only an intercept). – RScrlli Apr 9 at 8:52