What is the lasso in regression analysis? I'm looking for a non-technical definition of the lasso and what it is used for.
 A: LASSO regression is a type of regression analysis in which both variable selection and regulization occurs simultaneously. This method uses a penalty which affects they value of coefficients of regression. As penalty increases more coefficients are becomes zero and vice Versa. It uses L1 normalisation technique in which tuning parameter is used as amount of shrinkage. As tuning parameter increase then bias increases and as is decreases then variance increases. If it is constant then no coefficients are zero and as is tends to infinity then all the coefficients will be zero.
A: The LASSO (Least Absolute Shrinkage and Selection Operator) is a regression method that involves penalizing the absolute size of the regression coefficients. 
By penalizing (or equivalently constraining the sum of the absolute values of the estimates) you end up in a situation where some of the parameter estimates may be exactly zero. The larger the penalty applied, the further estimates are shrunk towards zero.
This is convenient when we want some automatic feature/variable selection, or when dealing with highly correlated predictors, where standard regression will usually have regression coefficients that are 'too large'.
https://web.stanford.edu/~hastie/ElemStatLearn/ (Free download) has a good description of the LASSO and related methods.
A: In "normal" regression (OLS) the goal is to minimize the residual sum of squares (RSS) in order to estimate the coefficients
$$
\underset{\beta \in \mathbb{R}^p}{\operatorname{argmin}} \sum_{i=1}^{n} (Y_{i} - \sum_{j=1}^{p}X_{ij}\beta_{j})^{2}
$$
In case of LASSO regression you estimate the coefficients with a slightly different approach:
$$
\underset{\beta \in \mathbb{R}^p}{\operatorname{argmin}} \sum_{i=1}^{n} (Y_{i} -
 \sum_{j=1}^{p}X_{ij}\beta_{j})^{2} \color{red}{+ \lambda \sum_{j=1}^{p}|\beta_{j}|}
$$
The new part is highlitened in red, which is a sum of the absolute coefficient values penalized by $\lambda$, so $\lambda$ controls the amount of (L1) regulazation.
Note that if $\lambda = 0$, it would result into same coefficients as that of Simple Linear Regression. The formula shows that in case of LASSO $\operatorname{argmin}$ needs both, RSS and L1 regulazation (new red part) to be minimal. If $\lambda = 1$, the red L1 penalty constrains the size of the coefficients so that the coefficient can only increase if this lead to the same amount of decrease in RSS. More generally, the only way the coefficients can increase is if we experience a comparable decrease in the residual sum of squares (RSS). Thus, the higher you set $\lambda$ the more penalty is applied to the coefficients and the smaller will be the coefficients, some might become zero. That means LASSO can result in parsimonious models by doing feature selection and it prevents the model from overfitting. That said, you can use LASSO if you have many features and your goal is rather to predict data than to interpret the coefficients of your model.
