Discrete and continuos parameters in MCMC sampler

Question

I'm working with a 6-dimensional Bayesian model, and the affine-invariant sampler implemented in emcee. Four of those parameters are discrete, while the other two are continuous.

emcee will propose continuos values for all the parameters as the next step in the sampler. The way I currently handle this is to "push" the values for the discrete parameters towards the closest "valid" values (ie: those in the discrete set), before passing the new step to the likelihood evaluation.

For example, assuming the first four parameters are the discrete ones, if the next step proposed by emcee is:

$A=(0.26234, 12.5567, 0.00544, 9.56, 0.4674, 1.333)$

I will change this to the "pushed" proposed step:

$A_p=(0.26, 12.56, 0.005, 9.6, 0.4674, 1.333)$

where the first four parameters are now "valid", and then evaluate the likelihood.

Is this a proper approach or am I messing up my samples? Are there other approaches? I also find that my chains (emcee works with multiple parallel chains) don't mix. The mean acceptance rate for all chains is below 1% and they can be seen stuck at a single value for almost their entire length. Could this be causing this problem?

Answer 1

EDIT: The validity of this approach depends on whether your next proposal begins with the rounded values or the non-rounded values.

New answer assuming that your next proposal uses the non-rounded values

Yes, this approach is valid. It can be formalized as follows. In place of a likelihood $L(X | \theta )$ and prior $\pi_\theta(\theta)$ where $\theta$ is discrete, define a new likelihood $Pr(X | \phi) \equiv Pr(X|R(\phi))$, where $\phi$ is continuous and $R$ rounds every entry to the nearest valid value. Run your sampler on $\phi$.

N.B. Some people might have the intuition that the prior on $\theta$ corresponding to the above scheme is to evaluate $\pi_\phi$ at $\theta$. It's not. A given prior $\pi_\phi(\phi)$ induces the following prior on $\theta$: $$\pi_\theta(\theta) = \int_{[\theta \pm 0.5]} \pi_\phi(z)dz$$.

Old answer assuming that your next proposal begins with the rounded values

This is not a valid approach. This affine-invariant sampler is a Metropolis-Hastings variant, which means the proposal distribution appears in two different spots in the algorithm. It appears in generating proposals, of course, and you've altered that spot. But the PDF of the proposal also is used when deciding whether to accept or reject the current proposal. If you change it in one place and not the other, there's no guarantee what will happen.

(EDIT: Even if you were to evaluate the density of the proposal correctly, it may break the symmetry property $g(1/z) = zg(z)$ that the sampler relies upon to achieve detailed balance.)

There's another issue here: the "stretch move" proposal used by this affine-invariant sampler uses a discrete proposal distribution! Each walker selects another walker and moves a certain percentage towards or away from that other walker. The percentage is fixed; it's a tuning parameter of the algorithm. I can't say for a fact that this is your problem, but it's easy to imagine a scenario where the step size is small, and your rounding drags it back to the current value no matter what. Could this be causing your problem? Yes.

You may be wondering what to do next, but that's a matter for another question:

What MCMC algorithms/techniques are used for discrete parameters?