How to apply a soft coefficient constraint to an OLS regression? I would like to estimate an ordinary least squares regression of the form
$$
y = X\beta + \varepsilon, \
$$
except that, instead of minimizing the sum of squared residuals,
$$
SSR(b)=(y-Xb)'(y-Xb)
$$
I want to minimize
$$
(y-Xb)'(y-Xb)+\lambda(b-\tilde{\beta})'(b-\tilde{\beta})
$$
where $\lambda$ is some constant.  Otherwise, notation above is as on wikipedia.  All the standard assumptions are satisfied.
Is there some way I can modify the regression to perform the joint minimization?
 A: Differentiating the objective function with respect to $b$ and equating to $0$ shows that the solution to the modified equation is obtained by solving
$$(X'X + \lambda)b = X'y + \lambda\tilde{\beta}.$$
If your software won't do that directly, you can get the same results with this trick:


*

*Include a column of 1's in the dataset to explicitly model the constant.  Do the fitting without a constant term.

*For $p$ independent variables (including the constant), include $p$ additional fake data.  For fake case $i$, $i=1,\ldots,p$, set $X_i = \sqrt{\lambda}$, $y = \sqrt{\lambda}\tilde{\beta}$, and all other $X_j=0$.  (Of course we require $\lambda \ge 0$.)
Although you can obtain the solution $\hat{b}$ this way, I doubt any of the statistics coming out of this fit will be meaningful.
A: This looks a lot like ridge regression, the lm.ridge function in the MASS package for R does ridge regression and the ols function in the MASS package also does penalized regression.  If neither of those does exactly what you want they could be used as a starting point.  You could also look at the lasso and lars algorithms (there are packages for these for R as well) which uses an L1 penalty term instead of L2.
