Looking for an intuitive explanation, thanks.
1
-
5$\begingroup$ Possible duplicate of Why is ridge regression called "ridge", why is it needed, and what happens when $\lambda$ goes to infinity? or stats.stackexchange.com/questions/118712/… $\endgroup$– Sycorax ♦Commented Mar 1, 2019 at 22:05
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
take the case of two perfectly correlated independent variables, x1 and x2 then the corresponding coefficients w1, w2 can go to +/- infty (by adjusting the other appropriately), and we have an infinite number of solutions.
adding L2 regularisation, means that of all these solutions (with same mean square error), there is a best solution - namely the one with smallest l2 norm. assuming we normalise the variables, this will have w1=w2, ie we take the average x1 and x2.
