# Regularized linear model: adding special constraints to the coefficient

I understand we can add $L_1$ or $L_2$ regularization to linear regression (Lasso and Ridge regression). In addition, it is possible to restrict the coefficient to be integers (see this post).

However, is there any related work to add special constraints to enforce the relationship between features?

• For example, suppose, I know feature 1 is much important than feature 2, so I want to make $\beta_1 \in [10,20]$ and $\beta_2 \in [1,2]$ as a constraint in the model.
• Another example would be, I think feature 1 is similar to feature 2. So, I want $|\beta_1-\beta_2|<c$.

If there is related work, please provide a link to the paper.
If not, how do people deal with incorporating domain knowledge about the importance of the feature to the model coefficient?

• This is a standard aspect of all linear models and has nothing to do with regularization. For example, a model based on $\beta_1 x_1 + \beta_2 x_2$ will (under the constraint) instead be based on $(2\beta_2)x_1 + \beta_2 x_2 = \beta_2(2 x_1 + x_2)$ which is still a linear model with one less parameter and a new "feature" constructed from $2x_1 + x_2$. Search our site: stats.stackexchange.com/search?q=regression+constraint – whuber Jul 20 '16 at 14:58
• Thanks @whuber, I think $\beta_1=2\beta_2$ is a bad example for what I was trying to ask. I revised my question. – Haitao Du Jul 20 '16 at 15:45

Another example would be, I think feature 1 is similar to feature 2. So, I want $|\beta_1-\beta_2|<c$.