Prior and posterior in Bayesian regression

Question

I am referring to the Bayesian inference in Simple linear regression as explained in Probability & statistics - Schervish/Degroot( 2012).

For a simple one variable regression, the authors assume an improper prior = 1/Tau. The posterior joint distribution of both beta coefficients is then the bi-variate normal, the posterior distribution of Tau is Gamma(2, n-2) ,& the marginal distribution of each Beta coefficient is the T(n-2).

I have some questions regarding this model:

1. Once the posterior calculations are done , and the appropriate confidence intervals for Beta and Tau are constructed, what happens if we have new information in the form of some new data points? In this case whether the prior would be the normal ? What would be the posterior in this case ?

2. Which approach would be better and why? a) Add the new data to the existing data & estimate the posterior based on complete data. b) Assume the posterior calculated earlier as a prior & then estimate the posterior ( Is it analytically possible to estimate the posterior)

Thanks Kedar

Answer 1

Because a Normal-Gamma prior was used and it is conjugate, the posterior would be Normal-Gamma.

This new posterior is the prior for the next points. You could add them to the prior dataset and calculate as if you had not performed the calculations already or you could treat the posterior as a prior and get a new posterior. The only exception to that rule is when the elements of the sample are not exchangeable. That is only an issue when the order of observation matters.

You would now have a proper prior if you use the posterior as your prior.

Unless the data is not exchangeable, it doesn't matter which way you do your calculations.

One last picky note, Bayesian intervals are called credible intervals or credible sets instead of confidence intervals. The distinction is made because they do not imply the same meaning and do not have the same types of mathematical requirements on them. For example, credible intervals can be disjoint sets.

Your textbook should have solutions for conjugate priors with slope and covariance matrix unknown.