1
$\begingroup$

I'm analyzing a simple linear regression $Y_{i}$~$a+b*X_{i}+e_{i}$, with $e$ being normally distributed with known variance and where I have normal priors on $a$ and $b$. I'm trying to piece together an approximate closed form solution for the variance of my posterior of $a$ as a function of sample size for the purposes of an optimization I'm trying to do.

I know that in general, there's a closed form conjugate prior for Bayesian regression in the multivariate case in matrix form. So I can do some sort of moment matching and then calculate out the matrix algebra for the bivariate case and probably can end up with a nice looking closed form. I've tried that, but I keep getting bogged in algebra, and I feel like this has to be a well-known problem that's been fleshed out already. Does anybody have a citation or link to point me to?

|cite|improve this question
$\endgroup$
  • $\begingroup$ Could you give us the full model specification and what you mean by 'basically know the variance' and 'approximately normal priors'? $\endgroup$ – user44764 Jul 24 '14 at 19:34
  • $\begingroup$ Clarified. Sorry for the loose language. $\endgroup$ – DavidShor Jul 24 '14 at 20:47
1
$\begingroup$

Please, take a look at Section 3.2.6 of the following document:

http://www.biostat.uzh.ch/teaching/master/previous/seminarbayes/SimonKunz_article.pdf

There, you will find the posterior of $(a,b)$ (which is a bivariate normal distribution in your case with a certain known variance structure) from where you can extract the marginal distributions, which are normal, and derive the corresponding variances.

Also, page 19 of the following document:

http://fisher.osu.edu/~schroeder.9/AMIS900/ech6.pdf

References

Bürgin, Reto: Bayesian linear model - Basics, Part 1. 2009

Gelman et al.: Bayesian data analysis, third edition.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Hi Jelinek, welcome to CV! Answers are better when they are standalone, since external links can sometimes changed or be removed altogether. Would you mind editing your answer to include the relevant material directly in it? $\endgroup$ – Patrick Coulombe Jul 24 '14 at 22:26
  • $\begingroup$ @PatrickCoulombe Thanks for your comment. The problem here is that 1. This problem is sort of well-known, as the OP guessed, and 2. There is A LOT of notation involved, which would require an afternoon of typing. I will add some references at the end, just in case the link is sent to Limbo. A basic google search returns several documents with the posterior of interest. $\endgroup$ – Jelinek Jul 24 '14 at 22:41

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.