0
$\begingroup$

I'm implementing Stochastic Search Variable Selection(SSVS) for logistic regression using R. I'm using Spike and Slab prior with indicators $\lambda_{j}$. Here's my model structure:
$Y ∼ Bernoulli(\mu(X))$
$log {μ(X)\over1−μ(X)} = X\beta$
$\beta_{j} \overset{ind}{∼}\lambda_{j}N(0,\sigma_1^2)+(1−λj)N(0,\sigma_2^2) \ for \ j=1,···,10$

$\lambda_{j} \overset{ind}{∼} Bernoulli(1/2)$
$\sigma_1^2 ∼ InverseGamma(1, 20), \ \sigma_2^2 ∼ Gamma(1, 20)$

And I will make $T$ MCMC samples of $\beta_j, \sigma_1^2, \sigma_2^2, \lambda_j$. When ${1\over T}\sum_{i=1}^T \lambda_{j,i} > 0.5$, predictor $X_j$ is selected.

But how can I sample $\lambda$? It's Bernoulli random variable, so its value is either 0 or 1. But for Metropolis - Hastings proposal, I need $q(\lambda'|\lambda)$, which is correlated with last sample. How can I correlate proposal of $\lambda'$ with current $\lambda$? Any help should be appreciated.

$\endgroup$

1 Answer 1

1
$\begingroup$

You have correctly identified that specifying $\lambda_i \sim \text{Bernoulli}(1/2)$ means that there's really no parameter and no updating here, i.e. there will always be 50% probability for the first mixture component and 50% for the second one.

One solution is to have an extra parameter e.g. like this: $\lambda_i \sim \text{Bernoulli}(\pi)$ with some prior (e.g. $\text{Beta}(0.5, 0.5)$ or something else) for $\pi$.

Note that applying the criterion $\frac{1}{T} \sum_{i=1}^T \lambda_{i,j} > 0.5$ is pretty arbitrary and the coefficients you estimate would be very different if you re-fit after selecting those covariates you select on this basis. It's a problem of post-selection inference, which tends to be problematic, if done in such a way that the variable selection is ignored.

By the way, you have a number of options in R (and other programming languages), where you would not have to code up your own sampler. E.g. the brms R package (that uses Stan in the background) implements has its own spike-and-slab prior implementation that has fat-tails via t-distributions (the horseshoe or regularized horseshoe prior - see the documentation). The link also offers further links to some of the papers on this that describe why this particular prior may be expected to have some good properties.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.