Algebra for logistic regression slice sampler

Question

I am having some difficulties when trying to do a little bit of algebra from Example 7.11 from the book "Introducing Monte Carlo Methods with R: Robert & Casella"

The example relates to logistic regression using a slice sampler. The model is (omitting $i$ index)

$$y \sim Bern(p(x)), \qquad p(x) = \frac{\exp(a + bx)}{1 + \exp(a + bx)}$$ And assuming flat priors on (a,b) the posterior distribution is given by

$$ p(a,b) = \prod_{i=1}^n \frac{\exp(y_i(a + bx_i))}{1 + \exp(a + bx_i)}$$

The first step of the slice sampler is given as

$$ U_i \sim\mathcal U \Bigg(0, \frac{\exp(y_i(a + bx_i))}{1 + \exp(a + bx_i)}\Bigg) $$ and the second as

$$ \mathcal U\left\{ (a,b): y_i(a+bx_i) > \log \frac{u_i}{1-u_i}\quad i=1,\ldots,n \right\} $$ I am struggling with this second step. I cannot get these values if I work backwards:

\begin{align} y(a+bx) &> \log u/(1-u) \\ \exp(y(a+bx)) &> u/(1-u) \\ (1-u)*\exp(y(a+bx)) &> u \\ \exp(y(a+bx)) &> u + u\exp(y(a+bx)) \\ \frac{\exp(y(a+bx))}{1 + \exp(y(a+bx))} &> u \\ \end{align}

The final term on the left-hand side does not equal the posterior distribution,whic I belive should be of the form $\{x: f(x) \geq U_i\}$. Where is my mistake please?

Answer 1

Sorry that this section proves a challenge! (I corrected the question to indicate that the Uniform on $(a,b)$ is imposing the inequality for all $i$'s, as this is (too) implicit in the book.) There is alas a typo in the book when defining the set (second displayed formula on p.220), not a mistake in your reasoning!

The constraint on $(a,b)$ is thus that for all $1\le i\le n$, $$\frac{\exp(y_i(a + bx_i))}{1 + \exp(a + bx_i)}\ge u_i$$ hence when $y_i=0$ $$u_i^{-1}\ge 1 + \exp(a + bx_i)$$ or $$\log\frac{1-u_i}{u_i}\ge a + bx_i$$ and when $y_i=1$ $$1-u_i\ge \frac1{1 + \exp(a + bx_i)}$$ or $$\log\frac{u_i}{1-u_i}\le a + bx_i$$ Meaning that $$ \max_{y_i=1}\left\{ \log\frac{u_i}{1-u_i}-bx_i\right\} \le a \le \min_{y_i=0} \left\{ \log\frac{1-u_i}{u_i}-bx_i\right\}$$ and the same for $b$. Thus the slice distributions (and the R code below) remain valid.

Apologies!!! I went back to our (LaTeX) source file for this chapter on Gibbs samplers and found that we had written

...variables. Generating a uniform distribution over the set $$ \bigcap_{i=1} \left\{ (a,b)\,:\ (-1)^{y_i}(a+bx_i) > \log\frac{u_i}{1-u_i} \right\} $$ being rather unwieldy, we can further decompose the uniform simulation by consecutively simulating...

Which is correct. For some reason I can neither fathom nor remember, both the intersection and power terms did not make it to the printed versions... (Including the French and Japanese translations!)