Skip to main content
1 of 3
Dan
  • 83
  • 4

Lasso regression Mathematical intuition

I do not fully understand lasso regression. I am able to code a LASSO regression model with sklearn, but I do not understand the Mathematical intuition behind the model, which I believe is important. In ISLR (Introduction to Statistical Learning in R), it is explained that the equation for lasso regression is RSS + lambda*sum of aboslute beta values (please forgive me. I do not know who to use latex to represent my equations properly). So, although I do not understand where exactly the hyperparameter comes from or what its actual purpose is, I do understand that the algorithm seeks to minimise that equation and reduce overfitting by introducing a little bias into the training fit.

However, it is shown later that the lasso regression equation can be rewritten as minimise(RSS) subject to sum of absolute beta less than or equal to s, which is where my confusion lies. Firstly, how can the original lasso regression equation be rewritten as that? Also, what is this s term? I get that it means sum, but I do not understand what sum this equation refers to or why it has to be less than or equal to the sum of the absolute beta values.

Again, please forgive me for my lack of LaTex skills, but whatever help you can give, I will be grateful.

Dan
  • 83
  • 4