I am learning why the L1 regulariser (Lasso) is used to encourage sparsity in ML models. When describing the proof, I am seeing that the regularised minimisation cost function;

$$ min(RSS(w) + \lambda* \| w\| ) $$

can be rewritten as a constrained but smooth objective;

$$ min(RSS(w))\;\;\;\;\; s.t. \; \| w\| \leq B $$

and that would mean we are optimising on the L1 unit circle.

I was hoping someone could help me understand why we can write the L1 as a constraint and what does that mean?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.