Bayesion priors in ridge regression with scikit learn's linear model I'm using scikit learn's linear model to do ridge regression.
Ridge regression penalizes parameters for moving away from zero. I want to penalize for moving away from a certain prior, with each parameter having a different prior. 
Is this possible with scikit learn's linear model? I know there's a BayesianRidge module there, but I'm not sure what it does.
 A: Ridge regression looks like:
$$
\min_{\beta}||Y-X\beta||^2 + \lambda_1 ||\beta||^2
$$
If you want to instead compute
$$
\beta^* = \arg\min_{\beta}||Y-X\beta||^2 + \lambda_1 ||\beta - \beta_0||^2
$$
I guess you could just turn this into shrinking towards zero using the new variable
$$\theta = \beta - \beta_0.$$
So you'd solve:
$$
\theta^*  := \arg\min_{\theta}||Y-X\beta_0-X \theta||^2 + \lambda_1 ||\theta||^2
$$
Then apply the change of variables again (i.e., $\beta^* := \theta^* + \beta_0$).
So to recap, if I have some black box function $\text{RidgeRegression}(Y,X, \lambda)$, I can use it to solve for an arbitrary prior $\beta_0$ simply by calling $\text{RidgeRegression}(Y-X\beta_0, X, \lambda)$.
A: What's posted in the only answer by Dapz does not do what it's supposed to do.
If I choose a value > 0 for any of the $\beta_0$, say the "i-th", the corresponding $\beta^*$ of "i" will be lower than with standard ridge regression, instead of higher as it should be (because we penalize for moving away from something > 0, instead of moving away from 0). 
A: I think this code should work, implementing the solution suggested by others above:
def fit_with_prior(model, X, y, sample_weight=None, prior=None):
    """Fit a regularized model with a nonzero prior"""
    assert prior is not None, "you need to specify a prior"
    new_y = y - np.sum(prior * X, axis=1)
    model.fit(X, new_y, sample_weight=sample_weight)
    model.coef_ += prior  # modifying underlying model's coefficients
    # what about the intercept?

Initialize the Ridge model object as you normally would: my_ridge = Ridge(alpha,...), and instead of calling my_ridge.fit(X, y), call fit_with_prior(my_ridge, X, y, prior=prior), where prior is a vector of length equal to the number of columns in x, being the prior values toward which you want to regularize. I think the intercept term is probably not penalized, and can thus be ignored for the purposes of this transformation (unless you explicitly added a column of constants to X, in which case it will be treated just like the other coefficients). I think it should also work for other regularized linear models, e.g. ElasticNet, as long as they use their coef_ attribute to do prediction and scoring, as Ridge seems to based on my testing.
