Is a spike and slab prior a proper prior?
(I am talking about a (product of Bernoulli) spike and Normal slab)
If not, does it still lead to a proper posterior?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
First let's look at Mitchell and Beauchamp (1988), for a description of what a spike-and-slab prior is:
That is, $\beta_j$ is uniformly distributed between the two limits $-f_j$ and $f_j$, except for a bit of probability mass concentrated at 0 if $x_j$ is vulnerable to deletion. We are interested in taking $f_j$ to be very large for all $j$, ...
Now if $f_j$ is large-but-finite, this is a proper prior - we can even write down the cdf explicitly.
(You'll sometimes see people actually draw something like this: - which might help with picturing it in some sense, but the problem with that is what then does the y-axis represent? It can't be density because the spike represents probability, and it can't be probability because the uniform part represents density. The two parts are on completely different scales. This seems to encourage the mistaken notion of conflating probability and density.)
Mitchell and Beauchamp keep $f_j$ finite but assume that it's large enough that the relevant integrals from $-f_j$ to $f_j$ can be well approximated by integrals from $-\infty$ to $\infty$.
If, however, we take the limit as $f_j\to\infty$ then of course it wouldn't be a proper prior. When being used as a prior for variable selection this generally isn't going to be done because of the way that impacts variable selection (try it for a simple case).
Other priors have been given the name "spike and slab" since -- including the case with a Gaussian slab, as you mention. In that case, the prior is proper as long as the variance of the normal is finite.
: Mitchell T.J. and Beauchamp, J.J. (1988),
"Bayesian Variable Selection in Linear Regression"
Journal of the American Statistical Association, Vol. 83, No. 404 (Dec.),1023-1032