What is the difference between "variables of interest" and "variables from which lasso selects" in Lasso? I'm getting started with regularization models and I notice that Lasso requires three inputs, a dependent variable and then two sets of what I assume as independent variables, one which are of interest and one from which the model selects.
What does this mean, exactly? I'm used to multiple regressions where you just insert the independent variables in one set.
 A: This distinction isn't always pre-specified in software, but it is often available (although this distinction might not be the default so that you have to read the manual to figure out how to do it). It's a way to limit regularization to predictors that control for additional influences on outcome while not penalizing coefficients of the predictors of primary interest.
The main point is that you don't need to penalize all the regression coefficients in a regularized model. That's true both for LASSO and for ridge regression.
For example, say that you have some predictors of primary interest and want to use other predictors primarily to control for additional influences on outcome. But you have too many predictors to incorporate without overfitting, based on the number of cases or events. In that situation you could choose not to penalize the predictors of primary interest and restrict penalization to coefficients of the predictors used to control for additional influences on outcome.
This paper illustrates the approach.
