I'm trying to use Bayesian inference to fit and interpret a linear model of the following form:

$$ y=X\beta + \epsilon \hspace{1cm} \text{where } \epsilon_i \sim \mathcal{N}(0,\sigma^2)$$

The parameters $\beta$ have a prior $\beta \sim \mathcal{N}(0,\tau=\gamma A^TA)$, for given matrix $A$ and hyperparameter $\gamma$.

To find appropriate hyperparameter values, I first calculate the marginal likelihood for $y$, $P(y|X,\gamma,\sigma^2)$ by integrating over parameters $\beta$ to obtain a $\mathcal{N}(0,XA^{-1}A^{-T}X^T/\gamma + I \sigma^2)$ distribution. I then maximise the pdf of this distribution to obtain optimal hyperparameter values $\hat{\gamma}$, $\hat{\sigma}^2$.

I want to interpret my point estimates by computing a 95% credible interval for the parameter $\gamma$, however, I'm a bit unsure how to go about this. My understanding was that to find credible intervals I need to find the distribution $P(\gamma|X,y,\sigma^2)$, however, I'm not sure what form this would take. I thought it could be calculated by applying Bayes' again, however I'm not sure how I would determine/compute the hyper-prior and evidence in this case. I'm not sure it would be tractable to calculate?

Any pointers on how I could find credible intervals would be much appreciated.

  • $\begingroup$ You are on the right track. You correctly noted that the thing you're missing is a prior for gamma. There is no way around this. If you really have no idea what the prior should be, this could be an argument for using a very flat prior, something like a gamma distribution with a variance of 1000, or some number that is large relative to the scale of your data. $\endgroup$ – zkurtz Apr 20 '18 at 3:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.