I am trying to implement Bayes Factor for model and variable selection in Bayesian Linear Regression and finding out corresponding type I and type II error. I need your help regarding this. I will be highly grateful if anybody can help me in this regard. For a linear regression model, $$\mathbf{y} = \mathbf{X}\boldsymbol{\beta} + \epsilon$$ with $\epsilon \sim \mathcal{N}(0, \gamma^{-1}\mathbf{I})$ and $\boldsymbol{\beta}\sim \mathcal{N}(\boldsymbol{\mu}_0, \gamma^{-1} \mathbf{V}_0^{-1})$ and $\gamma\sim Ga(a_0, b_0)$.

Firstly I am setting up hypothesis that $$H_0:\text{all} ~\beta = 0~\text{assumed as}~ M_0$$ against $$H_1: \text{few} ~\beta\neq 0~\text{assumed as}~ M_1$$

Bayes Factor $$B_{01} = \frac{p(D|M_0) p(M_0)}{p(D|M_1) p(M_1)}=\frac{p(\mathbf{y}|\boldsymbol{\beta}, \gamma) p(\boldsymbol{\beta}|\gamma)p(\gamma)}{\int\int p(\mathbf{y}|\boldsymbol{\beta}, \gamma) p(\boldsymbol{\beta}|\gamma)p(\gamma)d\boldsymbol{\beta}d\gamma}$$ assuming equal model prior probability $p(M_0) = p(M_1)$.

Now my question is: in numerator should I directly put the values of $\beta$ as $0$ and compute the value? Secondly, I have found out the the posterior on $\beta$ and $\gamma$ using Variational Inference, so if for the denominator closed form is available then can I directly put the posterior of those values in the expression?

Next, while calculating type I and type II error, I was going through the lecture material of Prof Berger, and found that $$p(\text{Type I error}) = \frac{B_{01}}{1 + B_{01}}$$ and $$p(\text{Type II error}) = \frac{1}{1 + B_{01}}$$ Are these valid formulae in case of such regression modeling?

Lastly, as I have computed the lower bound for marginal likelihood which is an essential component of Variational Inference, can this lower bound be used as marginal likelihood in the denominator?

I will be highly grateful to you if can kindly throw some light on this.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.