I'm implementing Stochastic Search Variable Selection(SSVS) for logistic regression using R. I'm using Spike and Slab prior with indicators $\lambda_{j}$. Here's my model structure:
$Y ∼ Bernoulli(\mu(X))$
$log {μ(X)\over1−μ(X)} = X\beta$
$\beta_{j} \overset{ind}{∼}\lambda_{j}N(0,\sigma_1^2)+(1−λj)N(0,\sigma_2^2) \ for \ j=1,···,10$

$\lambda_{j} \overset{ind}{∼} Bernoulli(1/2)$
$\sigma_1^2 ∼ InverseGamma(1, 20), \ \sigma_2^2 ∼ Gamma(1, 20)$

And I will make $T$ MCMC samples of $\beta_j, \sigma_1^2, \sigma_2^2, \lambda_j$. When ${1\over T}\sum_{i=1}^T \lambda_{j,i} > 0.5$, predictor $X_j$ is selected.

But how can I sample $\lambda$? It's Bernoulli random variable, so its value is either 0 or 1. But for Metropolis - Hastings proposal, I need $q(\lambda'|\lambda)$, which is correlated with last sample. How can I correlate proposal of $\lambda'$ with current $\lambda$? Any help should be appreciated.


1 Answer 1


You have correctly identified that specifying $\lambda_i \sim \text{Bernoulli}(1/2)$ means that there's really no parameter and no updating here, i.e. there will always be 50% probability for the first mixture component and 50% for the second one.

One solution is to have an extra parameter e.g. like this: $\lambda_i \sim \text{Bernoulli}(\pi)$ with some prior (e.g. $\text{Beta}(0.5, 0.5)$ or something else) for $\pi$.

Note that applying the criterion $\frac{1}{T} \sum_{i=1}^T \lambda_{i,j} > 0.5$ is pretty arbitrary and the coefficients you estimate would be very different if you re-fit after selecting those covariates you select on this basis. It's a problem of post-selection inference, which tends to be problematic, if done in such a way that the variable selection is ignored.

By the way, you have a number of options in R (and other programming languages), where you would not have to code up your own sampler. E.g. the brms R package (that uses Stan in the background) implements has its own spike-and-slab prior implementation that has fat-tails via t-distributions (the horseshoe or regularized horseshoe prior - see the documentation). The link also offers further links to some of the papers on this that describe why this particular prior may be expected to have some good properties.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.