7
$\begingroup$

Is a spike and slab prior a proper prior?

(I am talking about a (product of Bernoulli) spike and Normal slab)

If not, does it still lead to a proper posterior?

$\endgroup$
  • 5
    $\begingroup$ What makes you think otherwise? $\endgroup$ – Momo Mar 21 '15 at 23:56
  • 2
    $\begingroup$ Please add your definition of a spike-and-slab prior. $\endgroup$ – Xi'an Mar 22 '15 at 6:52
12
$\begingroup$

First let's look at Mitchell and Beauchamp (1988)[1], for a description of what a spike-and-slab prior is:

That is, $\beta_j$ is uniformly distributed between the two limits $-f_j$ and $f_j$, except for a bit of probability mass concentrated at 0 if $x_j$ is vulnerable to deletion. We are interested in taking $f_j$ to be very large for all $j$, ...

Now if $f_j$ is large-but-finite, this is a proper prior - we can even write down the cdf explicitly.

spike and slab prior cdf

(You'll sometimes see people actually draw something like this: not density - which might help with picturing it in some sense, but the problem with that is what then does the y-axis represent? It can't be density because the spike represents probability, and it can't be probability because the uniform part represents density. The two parts are on completely different scales. This seems to encourage the mistaken notion of conflating probability and density.)

Mitchell and Beauchamp keep $f_j$ finite but assume that it's large enough that the relevant integrals from $-f_j$ to $f_j$ can be well approximated by integrals from $-\infty$ to $\infty$.

If, however, we take the limit as $f_j\to\infty$ then of course it wouldn't be a proper prior. When being used as a prior for variable selection this generally isn't going to be done because of the way that impacts variable selection (try it for a simple case).

Other priors have been given the name "spike and slab" since -- including the case with a Gaussian slab, as you mention. In that case, the prior is proper as long as the variance of the normal is finite.

[1]: Mitchell T.J. and Beauchamp, J.J. (1988),
"Bayesian Variable Selection in Linear Regression"
Journal of the American Statistical Association, Vol. 83, No. 404 (Dec.),1023-1032

$\endgroup$
  • 3
    $\begingroup$ Re notation, I've seen the spikes notated on a pdf graph using arrows pointing upwards--so a spike with 0.5 probability would be an arrow up to $y = 0.5$. This is apparently the same notation physicists use for a Dirac delta function (which makes sense, since that's essentially what a spike pdf is), and helps distinguish the probability from the density as well. $\endgroup$ – Ben Kuhn Mar 22 '15 at 6:41
  • $\begingroup$ Can someone provide a simple example of this? I'm pretty confused. $\endgroup$ – O.rka May 15 '16 at 1:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.