This question is related to the another thread that I posted: Help with Variational Bayes on a weighted linear regression model

To reiterate, I have the model as follows:

$$ y_i \sim \mathcal{N}(T(x_i; \beta), \frac{\sigma^2}{w_i} {\textbf{I}}) \\ \beta \sim \mathcal{N} (\beta_{0}, \Sigma_{0}) \\ w_i \sim \mathcal{G} (a, b) \\ $$

The full joint model can be written as:

$$ p(\beta, w, y |x) = \prod_{i=1}^{N} \bigg[p(y_i|x_i, w_i, H) \ p(w_i)\bigg] \ p(\beta) $$

I suspect that using the Gamma distribution as a prior distribution for the weights $w_i$ is not opportunistic in the conjugate setting as I am unable to find the variational updates which allow the posterior distribution over $w_i$ being written in an analytical form as I am trying in that post.

So, my question is what distribution should I use for the prior on $w_i$ so that the posterior distribution after applying mean field and variational bayes inference comes out in some analytical form.

[EDIT]: The post @Xi'an pointed out deals with the so called g-prior but that example again does not try and estimate the weights of the regression problem in a Bayesian setting as far as I can tell.

  • $\begingroup$ Typically weights are known rather than estimated while $\sigma^2$ is estimated. $\endgroup$
    – jaradniemi
    Commented Oct 25, 2015 at 23:29
  • $\begingroup$ In my case, I am trying to estimate the weights. $\endgroup$
    – Luca
    Commented Oct 26, 2015 at 7:28
  • 1
    $\begingroup$ @Xi'an So should the prior on the weights parameter by modelled by this g-prior? I will try and work it out. However, in my case, the prior is on the weights of the regression. $\endgroup$
    – Luca
    Commented Oct 26, 2015 at 7:37
  • $\begingroup$ Right, I did not read carefully enough so retracted my vote. $\endgroup$
    – Xi'an
    Commented Oct 26, 2015 at 8:13
  • $\begingroup$ What is the form of $T(x,\beta)$? $\endgroup$
    – Xi'an
    Commented Oct 26, 2015 at 8:14


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.