# Bayesion priors in ridge regression with scikit learn's linear model

I'm using scikit learn's linear model to do ridge regression. Ridge regression penalizes parameters for moving away from zero. I want to penalize for moving away from a certain prior, with each parameter having a different prior.

Is this possible with scikit learn's linear model? I know there's a BayesianRidge module there, but I'm not sure what it does.

Ridge regression looks like:

$$\min_{\beta}||Y-X\beta||^2 + \lambda_1 ||\beta||^2$$

If you want to instead compute

$$\beta^* = \arg\min_{\beta}||Y-X\beta||^2 + \lambda_1 ||\beta - \beta_0||^2$$

I guess you could just turn this into shrinking towards zero using the new variable

$$\theta = \beta - \beta_0.$$

So you'd solve:

$$\theta^* := \arg\min_{\theta}||Y-X\beta_0-X \theta||^2 + \lambda_1 ||\theta||^2$$

Then apply the change of variables again (i.e., $\beta^* := \theta^* + \beta_0$).

So to recap, if I have some black box function $\text{RidgeRegression}(Y,X, \lambda)$, I can use it to solve for an arbitrary prior $\beta_0$ simply by calling $\text{RidgeRegression}(Y-X\beta_0, X, \lambda)$.

• Unless I'm missing something, $\text{RidgeRegression}(Y - X \beta_0, X, \lambda)$ gives you $\theta^*$, but you want $\beta^*$, so you actually want to replace $\text{RidgeRegression}(Y, X, \lambda)$ with $\text{RidgeRegression}(Y - X \beta_0, X, \lambda) + \beta_0$. – Dougal Jun 30 '14 at 0:30

What's posted in the only answer by Dapz does not do what it's supposed to do. If I choose a value > 0 for any of the $\beta_0$, say the "i-th", the corresponding $\beta^*$ of "i" will be lower than with standard ridge regression, instead of higher as it should be (because we penalize for moving away from something > 0, instead of moving away from 0).

• can you post some reproducible code? – Cam.Davidson.Pilon Jan 11 '13 at 19:49
• Did you intend to write this as a comment beneath Dapz's reply? Also, I've merged your two accounts but you will need to register this one to to benefit from all the advantages of the Stack Exchange community. – chl Jan 11 '13 at 22:26