I would like to estimate an ordinary least squares regression of the form $$ y = X\beta + \varepsilon, \ $$ except that, instead of minimizing the sum of squared residuals, $$ SSR(b)=(y-Xb)'(y-Xb) $$ I want to minimize $$ (y-Xb)'(y-Xb)+\lambda(b-\tilde{\beta})'(b-\tilde{\beta}) $$ where $\lambda$ is some constant. Otherwise, notation above is as on wikipedia. All the standard assumptions are satisfied.
Is there some way I can modify the regression to perform the joint minimization?
Differentiating the objective function with respect to $b$ and equating to $0$ shows that the solution to the modified equation is obtained by solving
$$(X'X + \lambda)b = X'y + \lambda\tilde{\beta}.$$
If your software won't do that directly, you can get the same results with this trick:
Include a column of 1's in the dataset to explicitly model the constant. Do the fitting without a constant term.
For $p$ independent variables (including the constant), include $p$ additional fake data. For fake case $i$, $i=1,\ldots,p$, set $X_i = \sqrt{\lambda}$, $y = \sqrt{\lambda}\tilde{\beta}$, and all other $X_j=0$. (Of course we require $\lambda \ge 0$.)
Although you can obtain the solution $\hat{b}$ this way, I doubt any of the statistics coming out of this fit will be meaningful.
This looks a lot like ridge regression, the lm.ridge function in the MASS package for R does ridge regression and the ols function in the MASS package also does penalized regression. If neither of those does exactly what you want they could be used as a starting point. You could also look at the lasso and lars algorithms (there are packages for these for R as well) which uses an L1 penalty term instead of L2.
