# Bayesian lasso vs spike and slab

Question: What are the advantages/disadvantages of using one prior over the other for variable selection?

Suppose I have the likelihood: $$y\sim\mathcal{N}(Xw,\sigma^2I)$$ where I can put either one of the priors: $$w_i\sim \pi\delta_0+(1-\pi)\mathcal{N}(0,100)\\ \pi=0.9\,,$$ or: $$w_i\sim \exp(-\lambda|w_i|)\\ \lambda \sim \Gamma(1,1)\,.$$

I put $\pi=0.9$ to emphasize most of the weights are zero and a gamma prior on $\lambda$ to pick the 'regularizing' parameter.

However, my professor keeps insisting that the lasso version 'shrinks' the coefficients and is not actually doing proper variable selection, i.e. there is an over-shrinkage of even the relevant parameters.

I personally find implementing the Lasso version easier since I use variational Bayes. In fact the Sparse Bayesian Learning paper which effectively puts a prior of $\frac{1}{|w_i|}$ gives even sparser solutions.

• Your professor is correct that it is shrinking relevant parameters, but so what? It only shrinks them to the extent that they are not contributing significantly to reducing the error. And why be focused on doing proper variable selection.. Shouldn't the focus be on reducing (test) error – seanv507 Dec 2 '15 at 7:27
• For most problems yes I would agree. However, for some problems (eg. cancer detection with gene expression) it is super important to find which features are the contributing factors. p.s. I've since moved on from my postdoc since he is a moron. Machine learning ftw!!! – sachinruk Dec 20 '16 at 23:14
• Spike and Slab happens to be the gold standard in variable selection and I also prefer to work with LASSO. @Sachin_ruk: the spike and slab prior can be implemented using Variational Bayes too... – Sandipan Karmakar Sep 15 '17 at 9:55
• @SandipanKarmakar could you post a link referring to spike and slab with Variational Bayes. – sachinruk Sep 17 '17 at 11:18
• Your question merges modelling [which prior?] and implementation [variational Bayes] issues. They should be processed separately. – Xi'an Oct 25 '17 at 5:17