6
$\begingroup$

I have a linear regression model $\bf Y=\bf{X}\bf{\beta}+\epsilon$. I want to assign a prior on $\bf\beta$ in order to derive the posterior predictive model $p(y_{predictive}|\bf{y},\bf{X},\beta)$. How do I decide which prior I assign to the regression coefficient $\bf{\beta}$ ? Is there a literature that discusses this?

$\endgroup$
  • 1
    $\begingroup$ The short answer is "use Zellner's g-prior". It is described in our book, Bayesian Essentials. $\endgroup$ – Xi'an Jan 26 '15 at 21:00
1
$\begingroup$

A nice document with general advice on choosing priors (with links to papers with more detail) is here: https://github.com/stan-dev/stan/wiki/Prior-Choice-Recommendations.

$\endgroup$

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

0
$\begingroup$

In practice, the prior that you select should come from domain expertise. Furthermore, a common practice is to select the prior based on your knowledge of the mean and then determining the variance based on your certainty.

For example, if your variables are economic variables for individuals (e.g. unemployed, level of education, GDP of country of residence, age) you might take hints from economic theory for predicting, for example, annual income; at the very least, you would at least have guidance for the signs of the means of the components of the $ \beta $ vector.

As for the variance, you would choose a higher variance for your prior if you're less sure about the effect of each component. You may also want to think about possible interaction effects between your variables (which would manifest in covariance between different $ \beta_i $ coefficients). Here it could be beneficial to impose additional assumptions, such as constant variance or a diagonal covariance matrix for your $ \beta $ prior, for computational simplicity.

At the end of the day it's kind of an art, but there are some cases for which you can obtain reasonable conjugate priors on $ \beta $ if you assume, for example that $ \beta $ has a multivariate Normal prior, and $ \epsilon $ is also some flavour of multivariate normal.

I would recommend Bayesian Data Analysis for more details on Bayesian Linear Regression.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.