How to apply a soft coefficient constraint to an OLS regression?

I would like to estimate an ordinary least squares regression of the form $$y = X\beta + \varepsilon, \$$ except that, instead of minimizing the sum of squared residuals, $$SSR(b)=(y-Xb)'(y-Xb)$$ I want to minimize $$(y-Xb)'(y-Xb)+\lambda(b-\tilde{\beta})'(b-\tilde{\beta})$$ where $\lambda$ is some constant. Otherwise, notation above is as on wikipedia. All the standard assumptions are satisfied.

Is there some way I can modify the regression to perform the joint minimization?

Differentiating the objective function with respect to $b$ and equating to $0$ shows that the solution to the modified equation is obtained by solving

$$(X'X + \lambda)b = X'y + \lambda\tilde{\beta}.$$

If your software won't do that directly, you can get the same results with this trick:

• Include a column of 1's in the dataset to explicitly model the constant. Do the fitting without a constant term.

• For $p$ independent variables (including the constant), include $p$ additional fake data. For fake case $i$, $i=1,\ldots,p$, set $X_i = \sqrt{\lambda}$, $y = \sqrt{\lambda}\tilde{\beta}$, and all other $X_j=0$. (Of course we require $\lambda \ge 0$.)

Although you can obtain the solution $\hat{b}$ this way, I doubt any of the statistics coming out of this fit will be meaningful.

This looks a lot like ridge regression, the lm.ridge function in the MASS package for R does ridge regression and the ols function in the MASS package also does penalized regression. If neither of those does exactly what you want they could be used as a starting point. You could also look at the lasso and lars algorithms (there are packages for these for R as well) which uses an L1 penalty term instead of L2.