Skip to main content
added 88 characters in body
Source Link
EdM
  • 101.5k
  • 11
  • 102
  • 303

This distinction isn't always pre-specified in software, but it is often available (although this distinction might not be the default so that you have to read the manual to figure out how to do it). It's a way to limit regularization to predictors that control for additional influences on outcome while not penalizing coefficients of the predictors of primary interest.

The main point is that you don't need to penalize all the regression coefficients in a regularized model. That's true both for LASSO and for ridge regression.

For example, say that you have some predictors of primary interest and want to use other predictors primarily to control for additional influences on outcome. But you have too many predictors to incorporate without overfitting, based on the number of cases or events. In that situation you could choose not to penalize the predictors of primary interest and restrict penalization to coefficients of the predictors used to control for additional influences on outcome.

This paper illustrates the approach.

This distinction isn't always pre-specified in software, but it is often available (although this distinction might not be the default so that you have to read the manual to figure out how to do it). It's a way to limit regularization to predictors that control for additional influences on outcome while not penalizing coefficients of the predictors of primary interest.

The main point is that you don't need to penalize all the regression coefficients in a regularized model. That's true both for LASSO and for ridge regression.

For example, say that you have some predictors of primary interest and want to use other predictors primarily to control for additional influences on outcome. But you have too many predictors to incorporate without overfitting, based on the number of cases or events. In that situation you could choose not to penalize the predictors of primary interest and restrict penalization to coefficients of the predictors used to control for additional influences on outcome.

This distinction isn't always pre-specified in software, but it is often available (although this distinction might not be the default so that you have to read the manual to figure out how to do it). It's a way to limit regularization to predictors that control for additional influences on outcome while not penalizing coefficients of the predictors of primary interest.

The main point is that you don't need to penalize all the regression coefficients in a regularized model. That's true both for LASSO and for ridge regression.

For example, say that you have some predictors of primary interest and want to use other predictors primarily to control for additional influences on outcome. But you have too many predictors to incorporate without overfitting, based on the number of cases or events. In that situation you could choose not to penalize the predictors of primary interest and restrict penalization to coefficients of the predictors used to control for additional influences on outcome.

This paper illustrates the approach.

Source Link
EdM
  • 101.5k
  • 11
  • 102
  • 303

This distinction isn't always pre-specified in software, but it is often available (although this distinction might not be the default so that you have to read the manual to figure out how to do it). It's a way to limit regularization to predictors that control for additional influences on outcome while not penalizing coefficients of the predictors of primary interest.

The main point is that you don't need to penalize all the regression coefficients in a regularized model. That's true both for LASSO and for ridge regression.

For example, say that you have some predictors of primary interest and want to use other predictors primarily to control for additional influences on outcome. But you have too many predictors to incorporate without overfitting, based on the number of cases or events. In that situation you could choose not to penalize the predictors of primary interest and restrict penalization to coefficients of the predictors used to control for additional influences on outcome.