1
$\begingroup$

I am trying to implement Bayes Factor for model and variable selection in Bayesian Linear Regression and finding out corresponding type I and type II error. I need your help regarding this. I will be highly grateful if anybody can help me in this regard. For a linear regression model, $$\mathbf{y} = \mathbf{X}\boldsymbol{\beta} + \epsilon$$ with $\epsilon \sim \mathcal{N}(0, \gamma^{-1}\mathbf{I})$ and $\boldsymbol{\beta}\sim \mathcal{N}(\boldsymbol{\mu}_0, \gamma^{-1} \mathbf{V}_0^{-1})$ and $\gamma\sim Ga(a_0, b_0)$.

Firstly I am setting up hypothesis that $$H_0:\text{all} ~\beta = 0~\text{assumed as}~ M_0$$ against $$H_1: \text{few} ~\beta\neq 0~\text{assumed as}~ M_1$$

Bayes Factor $$B_{01} = \frac{p(D|M_0) p(M_0)}{p(D|M_1) p(M_1)}=\frac{p(\mathbf{y}|\boldsymbol{\beta}, \gamma) p(\boldsymbol{\beta}|\gamma)p(\gamma)}{\int\int p(\mathbf{y}|\boldsymbol{\beta}, \gamma) p(\boldsymbol{\beta}|\gamma)p(\gamma)d\boldsymbol{\beta}d\gamma}$$ assuming equal model prior probability $p(M_0) = p(M_1)$.

Now my question is: in numerator should I directly put the values of $\beta$ as $0$ and compute the value? Secondly, I have found out the the posterior on $\beta$ and $\gamma$ using Variational Inference, so if for the denominator closed form is available then can I directly put the posterior of those values in the expression?

Next, while calculating type I and type II error, I was going through the lecture material of Prof Berger, and found that $$p(\text{Type I error}) = \frac{B_{01}}{1 + B_{01}}$$ and $$p(\text{Type II error}) = \frac{1}{1 + B_{01}}$$ Are these valid formulae in case of such regression modeling?

Lastly, as I have computed the lower bound for marginal likelihood which is an essential component of Variational Inference, can this lower bound be used as marginal likelihood in the denominator?

I will be highly grateful to you if can kindly throw some light on this.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.