# How do Machine Learning methods employ regularization to reduce variance?

I was reading a paper and came across this sentence:

"ML methods perform well by employing regularization to reduce variance and trading off regularization bias with overfitting in practice.:

I am wondering if anyone would have any insight into this sentence. I understand that regularization involves shrinkage of parameters, but I fail to see how shrinkage reduces variance and what it means to trade off the bias? Does anyone have any further insights here? Thanks.

• What do you mean by infinite L2 regularization? Is it when, say in a Lasso, $\lambda \to \infty$? – user321627 Feb 17 '18 at 19:48
• Like, if you added $\infty \sqrt{ \sum_i w_i^2}$ to your loss function (which would cause your weights to drop immediately to zero) – Hugh Perkins Feb 17 '18 at 19:51
• What is the $\infty$ symbol in front mean? – user321627 Feb 17 '18 at 19:52
• So, you would never use infinite L2 regularization. But yes, I mean something like $\lim_{\lambda \rightarrow \infty} \lambda \sqrt{\sum_i w_i^2}$ – Hugh Perkins Feb 17 '18 at 19:53