6
$\begingroup$

I searched about Bayesian Ridge Regression on Internet but most of the result i became is about Bayesian Linear Regression. I wonder if it's both the same things because the formula look quite similar

$\endgroup$
12
$\begingroup$

Ridge regression uses regularization with $L_2$ norm, while Bayesian regression, is a regression model defined in probabilistic terms, with explicit priors on the parameters. The choice of priors can have the regularizing effect, e.g. using Laplace priors for coefficients is equivalent to $L_1$ regularization. They are not the same, because ridge regression is a kind of regression model, and Bayesian approach is a general way of defining and estimating statistical models that can be applied to different models.

For reading more on ridge regression and regularization see the threads: Why does ridge estimate become better than OLS by adding a constant to the diagonal?, and What problem do shrinkage methods solve?, and When should I use lasso vs ridge?, and Why is ridge regression called "ridge", why is it needed, and what happens when $\lambda$ goes to infinity?, and many others we have.

$\endgroup$
  • $\begingroup$ Thanks for the answer ! i tried to understand what are the advantages of $L_2$ norm, the explanation on Scikit is a bit complicated for me. It would be nice if you could point out the problem with normal Ordinary Least Squares $\endgroup$ – Thien Feb 13 '18 at 10:43
  • 1
    $\begingroup$ @Thien see the edit for some links $\endgroup$ – Tim Feb 13 '18 at 10:51

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.