3
$\begingroup$

I'm learning Bayesian Linear Regression from a book, the linear model is $$p(w|x,\phi,\sigma^2)=Norm_w[\phi^Tx,\sigma^2]$$, as put in the book, we use Bayes approach to do the parameters estimation.

Here comes the problem: I thought it is pretty clear that we should introduce a conjugate prior for the parameters $\phi$ and $\sigma^2$, which should have a normal-scaled inverse gamma distribution, right? But the book first assume that $\sigma^2$ is known, and introduce a prior distribution for $\phi$ alone, which is a 0-mean Gaussian, and does the estimation for $\phi$. After all that, it assumes $\sigma^2$ is not known, and estimates it.

Why should we separate them?

UPDATE

The book is Machine Learning: A Probabilistic Perspective, p. 232, Section "Baysian Linear Regression."

I just found this article which also assumes one is known and later assumes it is unknown. Bayesian Linear Regression

$\endgroup$

1 Answer 1

2
$\begingroup$

Giving the author and title of the book would be helpful in deciphering the author's intention, since readers here might have read it.

But based on this information, it would appear to simply be a teaching approach intended to simplify the problem for illustrative purposes. Rather than estimating both parameters at once, the author estimates them in turn to give readers a feel for how the procedure of Bayesian regression works.

Alternatively, the purpose could be to compare how inference on the parameters is influenced by the choice of priors on $\sigma^2$, compared to a scenario where $\sigma^2$ is assumed known.

$\endgroup$
7
  • $\begingroup$ I update the post, adding one article which just do the same thing as in my book. $\endgroup$
    – avocado
    Dec 4, 2013 at 13:39
  • $\begingroup$ Ok, that's interesting, but finding an article that does the same thing as the unknown book doesn't help us divine the intention of your book's author, unless, by coincidence, they applied the same method for the same reason. $\endgroup$
    – Sycorax
    Dec 4, 2013 at 13:42
  • $\begingroup$ The book is MLaPP, ;-) $\endgroup$
    – avocado
    Dec 4, 2013 at 13:50
  • $\begingroup$ Thanks for that addition -- I haven't read it myself, but other readers who have might be able to provide more insight. $\endgroup$
    – Sycorax
    Dec 4, 2013 at 13:53
  • $\begingroup$ As said in the book, it assumes $\sigma^2$ is known indeed for simplicity, but the prior will be different when assume $\sigma^2$ is know or not, so I don't see the point why should it make such assumption. $\endgroup$
    – avocado
    Dec 4, 2013 at 13:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.